Exchange Ranking
Top Exchanges
24h Volume Ranking
Popularity Ranking
Exchange BTC Balance
Proof of Reserves
Decentralized Exchanges
Funding Rate
Funding Heatmap
Liquidation Data
Max Pain
Long/Short Ratio
Whale L/S Ratio
Binance/Okex/Huobi L/S
Bitfinex Margin L/S
ETF Tracker
Solana ETF
XRP ETF
Hong Kong ETF
Bitcoin Treasuries
Crypto Reversal
Ethereum Reserves
HyperLiquid Wallet Analysis
Hyperliquid Whale Watch
Large Transactions
On-chain Movement
Bitcoin ROI
Stablecoin Market Cap
Options Analysis
News
Articles
Economic Calendar
Features
Wallet
Contract Calculator
Security
Collections
Watchlist
FollowingBillions of TikTok video metadata released open source: AI developers 'new fuel and legal gray area
A developer named hashfunction released about 5.6 billion pieces of metadata for public TikTok videos on the open source AI platform Hugging Face, for free download by the public. The dataset was released by the datasocial account and spans from July 2014 to October 2026.
It needs to be clear that this is metadata and not the video itself. Each row of data includes title, label, audio ID, screen text, TikTok Shop product and seller ID, as well as interactive indicators such as page views, likes, comments, shares, favorites and downloads. The entire dataset is 460 GB in size, stored in Parquet format, and divided by month.
Data content and source details
Some fields are derived from TikTok's own tags, such as whether the video is marked as "AI-generated" and whether TikTok removes it from the "For You Page". Because these old videos come from archives, the AI tag fields for many early videos are empty.
This dataset has no access threshold and can be obtained by anyone. As of the time of statistics, Hugging Face showed that there had been 1,181 downloads. In addition, the dataset can also be accessed online through relevant websites.
Why is this dataset receiving so much attention?
AI developers have urgent needs for such data, but TikTok's official channels for obtaining data are relatively narrow. The titles, tags, audio IDs and interaction counts of billions of posts are the basic material for training models to predict viral content or TikTok Shop's hot-selling products. They are also key data for language models to learn the writing style of Short Video.
TikTok's official data acquisition approach is mainly for researchers from qualified academic institutions (including universities in the United States, the European Economic Area, the United Kingdom, Canada, Switzerland and other places, as well as some non-profit organizations in the European Union), and requires the application and approval process. Under Section 3.4 of TikTok's U.S. Terms of Service, the use of automated software to extract data from the platform is prohibited unless TikTok has written approval.
Technical implementation and controversy
Datasocial's technical note states that they accessed TikTok's private mobile API by disguising the generated device identity as an Android phone, reverse engineering the signature request, and simulating a TLS handshake. It is said that the system collected 3.23 billion creator profiles, 5.94 billion videos and 2.8 billion comments in three weeks, without the need for login or account participation.
This large-scale data grabbing has triggered legal proceedings. In October 2025, Reddit sued Perplexity and three data grabbing companies, accusing them of conducting "industrial-scale" content harvesting for AI training. In July, a federal judge largely rejected the defendant's motion to dismiss the case. Although the case involved grabbing data through Google search results rather than directly through private mobile APIs, the DMCA allegations it contained suggest that SerpApi allegedly circumvented Google's anti-robot protection mechanisms.
Business Model and License Agreement
This free version also serves as a commercial display window. The license uses CC BY-NC 4.0 (Creative Commons Attribution-Non-Commercial Use 4.0 International License), and the card states that it is only suitable for research and personal use. Commercial use, creator information and daily updates are directed to the official website, and the crawled source code is sold separately.
Regarding the data itself, the line contains the title, the music used, and various check box marks. Although there is no direct "creator username" column in the dataset, a lot of information is provided, including titles, tags, people mentioned in the video, etc. In fact, user IDs are available in the datasocial dataset.
Currently, similar datasets already exist on the platform. Another set of 4.5 billion video datasets described as collected from the TikTok mobile API is also located on Hugging Face. The data is free for non-commercial use, but the code used to collect the data sells for $1,699.
