Free List Of 93 Million Websites
We've extracted and compiled a list of 93 million websites from the CommonCrawl.org project from a crawl ending in October 2023. This is a list of websites that are currently live or were recently live so are still in the common crawlers index.
The list is in a text file format with the websites line seperated. You will need an application such as WinRar that is able to decompress the archive.
The websites are trimmed down to their root domains. For example blog. or www. has been removed from the domain.
If you are planning on opening the file to view the contents you will need an application that is capable of handling large txt files such as EmEditor as it is nearly 2GB once the rar file is decompressed.
It is a mix of websites from all countries. Please note there is no filtering of any sort so it will include adult content and possibly other websites you may not want to visit.
If you are going to use this list for digital marketing we ask that you are respectful of the free resource CommonCrawl has provided and do not conduct any aggressive spam campaigns and so on.
What does the file look like?
