Extracting publicly available data using automated tools is generally legal in the US under current court rulings. But the answer changes significantly depending on what type of data you collect, whether you bypass access controls, and what you plan to do with it. If you are training an AI model on scraped content, the legal situation is actively contested in multiple courts right now. The law is moving fast. What was settled yesterday may look different by the end of 2026. 

If you have ever sat across a table from a client asking whether their data pipeline is going to land them in court, you understand why this question matters. At Outsourcebigdata, we work with companies across industries that rely on data for market intelligence, competitor analysis, AI model development, and operational decision-making. The legal question around data extraction has come up in nearly every serious project conversation we have had in the last two years, and the honest answer in 2026 is this: it depends, and it is more complicated than most blog posts will tell you. 

This guide is built from current court rulings, active litigation, EU regulatory enforcement, and the practical reality of running data extraction projects at scale. We are not going to give you a simple yes or no, because that would be doing you a disservice. What we will give you is the clearest, most current picture of where the law actually stands in 2026, what has changed, and how to protect yourself if you are building or running a data extraction operation. 

Traditional  scraping has been around since the early days of the internet. A developer writes a script using tools like Python with BeautifulSoup or Scrapy, points it at a page, and tells it exactly which HTML elements to pull. It works well right up until the website changes its layout, at which point the whole thing breaks and someone has to go back in and fix it manually. 

AI-powered data extraction works differently. Instead of following a rigid set of rules based on HTML structure, AI-driven tools use machine learning and natural language processing to understand the meaning of content on a page. The system recognises that a block of text is a product price, a review, or a job title not because it sits inside a specific CSS class, but because the model understands context. When a site redesigns its pages, an AI scraper adapts. When content appears in an image, computer vision can still read it. This is what makes AI extraction significantly more powerful, and significantly more capable of operating at a scale that catches the attention of lawyers and regulators. 

The legal picture matters here because the scale and purpose of AI extraction differ from scraping a handful of product prices. We are now talking about tools that can crawl millions of pages per day, extract personal information from social media at industrial volume, and feed entire corpora of copyrighted text into AI training pipelines. That shift in scale and purpose is exactly why 2026 is proving to be a turning point in how courts and regulators treat this activity. 

This is the question we hear most often. The short answer is that scraping publicly available data is generally legal in the United States under current case law. The longer answer is that it comes with important conditions, and those conditions matter enormously. 

The landmark case that most people reference is hiQ Labs versus LinkedIn. LinkedIn sent hiQ a cease-and-desist letter threatening action under the Computer Fraud and Abuse Act, arguing that scraping its user profiles constituted unauthorised access to its systems. The Ninth Circuit Court of Appeals disagreed. The court ruled that the CFAA’s prohibition on accessing a computer system “without authorisation” applies to systems protected by access controls, like passwords, not to data that is freely visible to anyone with a browser. The Supreme Court reinforced this view in Van Buren versus United States, narrowing the CFAA further by establishing that violating a website’s Terms of Service alone does not constitute a criminal violation of the act if the underlying data was publicly accessible. The same principle held in Meta versus Bright Data, where even Meta’s significant legal resources could not use the CFAA to stop a commercial scraper from collecting public profile data at scale. 

The practical conclusion from these rulings is clear: if you can see the data in a browser without logging in, collecting it automatically does not make you a hacker under US federal law. However, that legal protection evaporates the moment you do any of the following:

• Bypass authentication: Logging in to access data, using credentials that are not yours, or circumventing a paywall puts you squarely within CFAA territory. 
• Create fake accounts: hiQ’s final settlement was partly triggered by its use of fake LinkedIn accounts. This is a contract violation and potentially criminal. 
• Break through CAPTCHAs or IP blocks: Circumventing these technical measures is increasingly argued to be a violation of DMCA Section 1201, a separate and serious legal risk. 
• Extract data from gated or private systems: Any data that requires a credential, subscription, or explicit login to access is protected. 

At-a-Glance: Legal Status of Common Scraping Actions (United States) 

Many people assume that if a website’s Terms of Service say “no scraping”, then scraping is illegal. This is a common and understandable misconception, but the law is more nuanced than that. 

A Terms of Service agreement is a contract, not a law. Whether that contract is enforceable against you depends on whether you actually agreed to it. Courts distinguish between two types of ToS agreements: browsewrap agreements, where terms are buried in a footer link that you never explicitly clicked, and clickwrap agreements, where you actively ticked a box or clicked a button saying you agreed. Courts have generally been reluctant to hold scrapers to browsewrap terms, particularly for publicly visible data. But if you created an account and agreed to the ToS during that process, those terms become a binding contract. Violating them exposes you to civil claims for breach of contract, even if it does not trigger the CFAA. 

The practical risks of a ToS violation short of a court case are also real. Platforms actively monitor for scraping patterns and respond with IP bans, legal notices, and cease-and-desist letters. For companies like Outsourcebigdata that run data extraction projects at scale, respecting ToS and rate-limiting requests is not just a legal consideration, it is operational good practice. A scraper that triggers a platform’s defences is a scraper that stops delivering data. 

Yes. Significantly. This is the part of the legal picture that catches many teams off guard. The act of extracting data and the act of using it afterwards are governed by different bodies of law, and you need to be compliant with both. 

One of the most common mistakes teams make is assuming the law is the same everywhere. It is not, and the gaps between jurisdictions are significant enough to change your entire compliance approach depending on where your users or data subjects are located. 

Jurisdiction Comparison: AI  Data Extraction in 2026 

This is the section your competitors’ articles are most likely to have out of date. The legal landscape in 2026 looks materially different from 2024, and several developments have real implications for any organisation involved in data extraction.

Key Legal Developments: 2025 to 2026 

• Bartz v. Anthropic: $1.5 billion settlement over copyrighted books used for AI training 
• Reddit v. Anthropic (filed June 2025, ongoing): breach of contract, unjust enrichment, trespass to chattels 
• Reddit v. Perplexity (filed late 2025, ongoing): DMCA Section 1201 circumvention claims 
• NYT v. OpenAI and Microsoft: fair use in AI training, ruling expected late 2026 
• YouTube creators v. Nvidia, Snap, Meta: DMCA circumvention applied to video AI training 
• EU AI Act: full enforcement from August 2026 requiring training data disclosure 
• CCPA 2026 update: California residents can opt out of AI systems using their data

The Bartz versus Anthropic settlement is the clearest signal yet that the days of quietly downloading copyrighted works for training purposes are over. A $1.5 billion settlement is not a slap on the wrist. It tells every AI company with a training pipeline that the exposure is existential if they get the copyright question wrong. 

The Reddit cases are interesting for a different reason. Reddit’s lawsuits against Anthropic and Perplexity are deliberately structured to avoid copyright law and the CFAA. Instead, they use breach of contract, unjust enrichment, and trespass to chattels. This approach sidesteps the preemption arguments that have helped scrapers in CFAA cases, and it signals a new legal template that other platforms are watching closely. Reddit’s argument is simple: we built and maintain this platform, we own the relationship with our users, and we have demonstrated the value of this data through paid licensing deals with Google and OpenAI. If you want it, you pay for it. Reddit versus Perplexity goes further, alleging that Perplexity increased its scraping volume 40 times after receiving a cease-and-desist, and directly naming the scraping intermediaries it used. This matters because it puts the entire supply chain of data extraction under legal scrutiny, not just the end AI company. 

The EU AI Act coming into full enforcement in August 2026 is the other major shift. Article 53 requires General Purpose AI providers to have a compliant privacy policy and to respect machine-readable signals defining what they can and cannot do with site content. TDMRep, the W3C standard for embedding text and data mining opt-outs into HTTP headers, is now legally binding in the EU. Publishers and website owners who deploy it can expect legal recourse if AI crawlers ignore it. 

For most of the internet’s history, robots.txt has been treated as a technical convention rather than a legal requirement. It tells automated systems which parts of a site the owner prefers they do not access, but there has been no legal mechanism to enforce it in most jurisdictions. That is changing in 2026, and you need to know how. 

In the United States, ignoring robots.txt is not a criminal act, but courts increasingly treat disregard for it as evidence of bad faith. In trespass-to-chattels claims, where a platform argues that excessive scraping harmed its systems, ignoring technical signals like robots.txt strengthens the platform’s case that the scraper was acting recklessly. In the EU, the situation is more direct. Under Article 53 of the EU AI Act, GPAI providers are legally required to respect machine-readable opt-out signals. This brings robots.txt-style controls into legally binding territory for AI training specifically. 

Beyond robots.txt, two newer standards are emerging that data extraction teams need to be aware of: 

• ai.txt: A newer file type that lets website owners specify purpose-based permissions. A site might allow search engine indexing but explicitly prohibit AI training. This granular control is becoming the standard in 2026. 
• TDMRep (Text and Data Mining Reservation Protocol): The W3C standard that embeds permissions directly into HTTP headers. In the EU, this is now the gold standard for legally enforceable opt-outs. If your crawler ignores it, you are not just being impolite, you are potentially violating both IP and privacy law. 

Our recommendation at Outsourcebigdata: treat robots.txt, ai.txt, and TDMRep as legally relevant signals regardless of jurisdiction. The cost of respecting them is minimal. The cost of ignoring them is growing quickly. 

Personal data is where the most legal risk concentrates for most extraction projects, and it is the area where teams most consistently underestimate their obligations. The key principle that the EU established and that regulators globally are increasingly adopting is this: the fact that data is publicly visible does not remove your obligation to handle it within the law. 

If the data you are collecting can identify a living person, directly or indirectly, it is personal data under GDPR. That includes names, email addresses, phone numbers, location data, IP addresses, and even combinations of seemingly innocuous data points that together make someone identifiable. Profile data on LinkedIn, comments on Reddit, posts on social media, contact information on company websites, all of this qualifies. 

For EU data, you need a lawful basis under GDPR Article 6. For commercial scraping, the most commonly used basis is legitimate interest under Article 6(1)(f), but using this requires conducting a Legitimate Interest Assessment that documents your purpose, balances it against the individual’s rights, and demonstrates that your scraping would not override their reasonable expectations. You must also be ready to respond to data subject access requests and deletion requests. A scraping project that collects personal data and has no mechanism to honour deletion requests is not GDPR compliant, regardless of how the data was collected. 

Personal Data Compliance Checklist for Extraction Projects 

• Identify whether any collected data can identify a living person 
• Document your lawful basis before collection begins (legitimate interest is most common) 
• Conduct a Legitimate Interest Assessment for GDPR-covered data 
• Minimise data: only collect what the project actually needs 
• Set retention limits and a deletion schedule 
• Build a mechanism to honour data subject access and deletion requests 
• Review whether California CCPA 2026 opt-out rules apply to your dataset 

Yes. Courts and regulators in 2026 are increasingly treating AI model training as a distinct purpose that requires its own legal analysis, separate from general data collection. 

In the United States, the central legal battleground for AI training data is copyright fair use. The four-factor fair use test asks whether the use is transformative, whether it affects the market for the original work, how much of the work was used, and whether the work was commercially published. AI companies have argued that training is transformative because the model learns patterns rather than reproducing the original text. Plaintiffs including the New York Times have argued the opposite, presenting evidence of AI outputs that closely reproduce protected articles in ways that substitute for the original. The courts have not reached a consistent answer yet, and the most anticipated fair use decisions on AI training are expected in mid-to-late 2026. 

The emerging commercial reality is also significant. Reddit’s licensing deals with Google and OpenAI, reportedly worth $60 million and $70 million respectively, have established a market price for access to large-scale conversational data. When Reddit then sued Anthropic for scraping the same data without a licence, it was able to point to those deals as evidence that a licensing market exists. In copyright law, harm to an existing licensing market is one of the strongest factors against fair use. This is the argument that could ultimately shift how courts treat AI training data across the board. 

At Outsourcebigdata, when we scope AI training data collection projects, our standard advice is: factual, non-copyrighted, public data carries the lowest legal risk. Original articles, creative writing, user-generated content on platforms with active licensing programs, and source code all carry meaningful copyright exposure. For those categories, licensing from the content owner is increasingly the right business decision, not just the ethical one. 

Running a legally sound data extraction operation in 2026 is absolutely achievable. But it requires treating legal compliance as part of the operational design of your project, not as an afterthought once the pipeline is already running.