Lens Protocol BigQuery

Lens Protocol on BigQuery

Overview

Lens social graph is published to a public BigQuery dataset, allowing anyone to query data in bulk without building complex infrastructure or indexers. BigQuery provides a solution for bulk data retrieval essential for machine learning training, analytics, and data profiling.

All data in Bigquery datasets is public information from Lens Chain. Datasets are updated every 15 minutes to reflect the latest network activity. If you need real-time data, please use the Lens GraphQL API, accessing snapshot data from the PostgreSQL database by indexers.

With the Lens Protocol BigQuery dataset, you can build various applications, such as:

Analytics and Dashboards: Create analytics dashboards to visualize user engagement, content performance, and community growth metrics.
Machine Learning: Extract large data volumes for training ML models to discover content trends, predict user engagement, or recommend content.
Custom Feeds: Develop personalized content feeds with your own algorithms using post data, reactions, and user interactions.

Accessing the Data

To query the Lens Protocol public datasets from BigQuery, use these fully-qualified table names:

Mainnet: lens-protocol-mainnet.INSERT_YOUR_TABLE_NAME_HERE
Testnet: lens-protocol-testnet.INSERT_YOUR_TABLE_NAME_HERE

You MUST query this from the US region. If you try to use the EU region it will not be able to find it.

To access 10 posts on mainnet/testnet:

SELECT * FROM lens-protocol-mainnet.post.record LIMIT 10

Data Format

Some of the data in the dataset is stored in its raw binary format, e.g. address. When syncing with BigQuery, we transform this byte data into hexadecimal strings with the format \x{string}. Lens provide a public function called FORMAT_HEX that converts BigQuery's \x format to the standard Web3 0x format.

SELECT  account as original_account_address,  `lens-protocol-mainnet.app.FORMAT_HEX`(account) as web3_formatted_account_addressFROM `lens-protocol-mainnet.app.user`LIMIT 5;