# How to Scrape Reddit: Posts, Comments & Subreddits (No Login)

> Learn how to scrape Reddit posts, comments, subreddits and search results without coding, no Reddit login. 64 data fields per result, step by step.

- **Author**: Shehriar Awan
- **Published**: 2026-09-23 (23 Sept 2026)
- **Reading time**: 13 min
- **Canonical**: https://www.lobstr.io/blog/how-to-scrape-reddit

---

So here you are... trying to **scrape Reddit**, and every tutorial says the same thing.

"Just add `.json` to the end of any Reddit URL!"

You try it. You get a 403. 🙃

![Terminal showing a Reddit .json URL returning HTTP 403](https://d37gzvgyugjozl.cloudfront.net/scrapereddit_how_to_scrape_reddit_posts_comments_subreddits_b8e51a7d29.png "width=2160;height=582")

I tested it myself on 18 September 2026, with three different User-Agents. Same 403 every time, byte for byte identical.

Don't worry, I got you.

Here's a no-nerd tutorial on **how to scrape Reddit posts, comments and subreddits**, without coding and without logging in.

But hang on... why did the `.json` trick stop working in the first place?

## Does Reddit offer an official API

> Let me give it to you straight: **yes, but not one you can use without OAuth and a lawyer.**

Reddit used to let you append `.json` to any URL and get clean structured data back. No key, no account, nothing.

That ended on **28 May 2026**, when Reddit posted this in r/modnews.

![Reddit's r/modnews post announcing the shutdown of unauthenticated .json access](https://d37gzvgyugjozl.cloudfront.net/scrapereddit_does_reddit_offer_an_official_api_96f23f6777.png "width=1546;height=1304")

An admin clarified it the next day. **Requests without OAuth or user credentials get blocked.**

The first public breakage report landed on GitHub **less than a day later**. No migration window.

And here's the part that catches people out.

**Reddit's HTML pages still return HTTP 200.**

![Reddit HTML page returning HTTP 200 with js_challenge and jsc_token fields](https://d37gzvgyugjozl.cloudfront.net/scrapereddit_does_reddit_offer_an_official_api_2_b6a7ffede3.png "width=2160;height=551")

That 200 is an 8,411 byte JavaScript challenge page carrying `js_challenge` and `jsc_token` fields. Your scraper thinks it succeeded and parses exactly zero posts.

If you're checking `response.status_code == 200`, you will never notice.

**So what about the real API?**

The official Data API lives at `oauth.reddit.com`. It comes with strings attached:

* **100 queries per minute** per OAuth client ID on the free tier
* **A separate commercial agreement** for "commercial purposes, research in excess of rate limits, or any use that is not expressly permitted"
* **Listings cap out around 1,000 items**, and comment trees truncate into `more` stubs that each cost another request
* **Reddit is winding it down.** In August 2026 the CTO announced the Public Data API is being replaced by Devvit, with new requests gradually restricted

Oh, and `robots.txt`? Now `Disallow: /` for every crawler on earth. Not even Googlebot gets an exception.

![Reddit robots.txt disallowing every crawler](https://d37gzvgyugjozl.cloudfront.net/scrapereddit_does_reddit_offer_an_official_api_3_35a2eea93a.png "width=2328;height=622")

And that's exactly why you need a scraper.

But how do you actually scrape Reddit at scale?

## 2 ways to scrape Reddit

You've got two options:

1. Build your own scraper
2. Use a ready-made scraper

### Build your own scraper

This one's for the nerds. People who enjoy mass destruction.

You're not fighting a rate limit anymore. You're fighting a **JavaScript proof-of-work challenge** that only hands over a cookie once your code solves it.

![Three different User-Agents getting the same 403 from Reddit](https://d37gzvgyugjozl.cloudfront.net/scrapereddit_build_your_own_scraper_a8d98b15c8.png "width=2160;height=673")

Open source maintainers shipped a workaround within 48 hours, and called it "very fragile" in their own release notes.

They were right. **Reddit renamed the challenge fields in September 2026** and those workarounds broke again.

Here's what else you're signing up for:

* **A polite User-Agent buys you nothing**... a browser string and a descriptive `MyTestScript/1.0 by u/someuser` returned the identical 403
* **Slowing down doesn't help**, the block is stateless and you're refused on request number one with no `x-ratelimit` headers
* **`old.reddit.com` is not an escape hatch**, it never serves the challenge page at all
* **RSS feeds still work** but give you 25 items with no score, comment count, upvote ratio or flair, and one query parameter trips the 403

> **True story**: even Claude couldn't grab a Reddit screenshot for this article. Reddit hit its automated browsers with a CAPTCHA, then a flat block, so I took it by hand.

![Reddit blocking automated browsers with a reCAPTCHA, then a network security block](https://d37gzvgyugjozl.cloudfront.net/scrapereddit_build_your_own_scraper_2_3d9f5bb436.png "width=1592;height=568")

### Use a ready-made scraper

This is what smart people do.

Even here you've got APIs and no-code options. **APIs are for nerds.** This article is no-code only.

I've already done a **full scored comparison of the [best Reddit scrapers](https://www.lobstr.io/blog/best-reddit-scrapers)**. For now, here's the one I actually use.

### Best Reddit scraper: lobstr.io

[lobstr.io](https://www.lobstr.io) is a no-code cloud scraping platform with 50+ ready-made scrapers. One of them is the Reddit Scraper.

![lobstr.io Reddit Scraper store page](https://d37gzvgyugjozl.cloudfront.net/scrapereddit_best_reddit_scraper_lobstrio_a788d5fe7c.png "width=1440;height=640")

[Scrape any subreddit, post, or search in minutes](https://www.lobstr.io/store/reddit-scraper)

#### Features

* **No Reddit account required**... no login, no cookies, no browser extension, no OAuth app to register
* **64 data points** per result across posts, comments and subreddit metadata
* **Five input types** from one field... a post, a subreddit, a user profile, a global search or a subreddit search
* **Search inside comment bodies**, not just post titles, which is where people actually describe their problems
* **Engagement metrics calculated for you** like `score_per_hour`, `age_hours` and `comment_to_score_ratio`
* **Subreddit metadata** including rules and weekly active users as structured data
* **Bulk upload** of as many URLs as you want
* Schedule recurring runs for **automated monitoring**
* Export to CSV, Excel, JSON, JSONL, [Google Sheets](https://help.lobstr.io/integrations/google-sheets), [Amazon S3](https://help.lobstr.io/integrations/amazon-s3) or [email](https://help.lobstr.io/integrations/email)
* [API access](https://help.lobstr.io/getting-started/api-key) and an MCP server for developers

#### Data

```md

| 🔗 URL | 🆔 POST ID | 📝 TITLE | 📄 BODY |

| 🏷️ POST TYPE | ⬆️ SCORE | 💬 COMMENT COUNT | 🏆 AWARD COUNT |

| 👤 AUTHOR | 🆔 AUTHOR ID | 📌 SUBREDDIT | 📌 SUBREDDIT PREFIXED NAME |

| 🆔 SUBREDDIT ID | 🏷️ FLAIR | 🌐 DOMAIN | 🗣️ LANGUAGE |

| 📅 CREATED AT | 🧬 TYPE | 💬 COMMENT ID | ↩️ PARENT ID |

| 🪜 DEPTH | ⭐ KARMA | ⭐ POST KARMA | ⭐ COMMENT KARMA |

| 📊 CONTRIBUTIONS | 🏅 TROPHIES | 📋 ACTIVE SUBREDDITS | 🛡️ MODERATED SUBREDDITS |

| 📄 SUBREDDIT DESCRIPTION | 👥 WEEKLY ACTIVE USERS | 📈 WEEKLY CONTRIBUTIONS | 📜 SUBREDDIT RULES |

| 🔖 SUBREDDIT RESOURCES | 📊 UPVOTE RATIO | 🔒 LOCKED | 📦 ARCHIVED |

| 📌 STICKIED | 🔞 OVER 18 | ⚠️ SPOILER | 🎬 IS VIDEO |

| 🖼️ IS GALLERY | 🖼️ MEDIA URL | 🖼️ THUMBNAIL | 📄 BODY HTML |

| 📏 BODY LENGTH | 📄 COMMENT BODY HTML | ⏳ AGE HOURS | 🚀 SCORE PER HOUR |

| ⚖️ COMMENT TO SCORE RATIO | 🔥 IS HIGH ENGAGEMENT | 🎖️ DISTINGUISHED AS | 📢 IS PROMOTED |

| 📝 IS SELF | 🔗 SHORTLINK | ✏️ EDITED | ✏️ EDITED AT |

| 🔄 NUM CROSSPOSTS | 🏷️ AUTHOR FLAIR | 💎 AUTHOR PREMIUM | 🥇 GILDED |

| 🌱 IS ORIGINAL CONTENT | 👥 SUBREDDIT SUBSCRIBERS | 🔄 CROSSPOST PARENT ID | 🔀 SUGGESTED SORT |

```

Every row is typed. A `post` row fills 42 of those, a `comment` row fills 12, and a `subreddit` row carries the community metadata.

#### Pricing

![lobstr.io pricing plans](https://d37gzvgyugjozl.cloudfront.net/scrapereddit_pricing_043418516d.png "width=1090;height=638")

* 100 results free to start
* Starts at $2 per 1,000 results
* Drops to **$0.50 per 1,000 results at scale**

One result costs one [credit](https://help.lobstr.io/core-concepts/credits), with no add-on functions billed on top. Empty runs cost nothing... I ran one that returned zero rows and it consumed zero credits.

So how long does it take to set up?

## How to scrape Reddit using lobstr.io [Step by Step Guide]

Scraping Reddit with lobstr.io takes less than 2 minutes. Here's the process:

1. Create a Squid
2. Add tasks
3. Adjust behavior
4. Launch
5. Enjoy

### Step 1: Create a Squid

A Squid is just a scraper instance. To create one:

1. Log in to your lobstr.io dashboard
2. Click the red *New Squid* button
3. Type "Reddit" in the search bar
4. Select `Reddit Scraper`

![Creating a new Squid with the Reddit Scraper in lobstr.io](https://d37gzvgyugjozl.cloudfront.net/scrapereddit_step_1_create_a_squid_66a79476e7.jpg "width=1232;height=784")

That's it. Your Squid is ready to configure.

👉 [What is a Squid?](https://help.lobstr.io/core-concepts/squids)

### Step 2: Add tasks

A task is simply your input. Here, **it's any Reddit URL you want to scrape**.

The scraper auto-detects which kind you gave it. All five of these work:

```md

https://www.reddit.com/r/Bitcoin/
https://www.reddit.com/r/Bitcoin/comments/1rcbgsa/crypto_beginners/
https://www.reddit.com/user/thisisbillgates/
https://www.reddit.com/search/?q=bitcoin
https://www.reddit.com/r/Bitcoin/search/?q=lightning

```

Paste a URL in the *Url* field and hit *Add +*.

![Adding Reddit URLs as tasks in lobstr.io](https://d37gzvgyugjozl.cloudfront.net/scrapereddit_step_2_add_tasks_f65fdd3f89.png "width=1630;height=1224")

Got hundreds of subreddits to monitor? Save them in a TXT or CSV file and hit *Upload file* instead.

> **Pro tip**: comments only come from post URLs, the ones with `/comments/` in them. Feed it a subreddit URL and you'll get that subreddit's posts plus one metadata row, but no comments. This is the single most common mistake people make with this scraper.

Once your tasks are added, click *Save* to move on to Settings.

### Step 3: Adjust behavior

**The Settings tab controls how your scraper behaves.**

I'm not going to bore you with every option. Let me cover the ones you'll actually use.

![Reddit Scraper basic settings](https://d37gzvgyugjozl.cloudfront.net/scrapereddit_step_3_adjust_behavior_9274c474f6.webp "width=1322;height=2000")

**Newer than**... stops the scraper at content older than your threshold, relative or absolute.

**Subreddit Sort Order**... controls which posts you get. Defaults to `hot`.

**Sort Period**... sets the time window when you sort by `top` or `controversial`: all time by default, or the past year, month, week, day or hour.

**Search Keyword**... drop a keyword here alongside a subreddit URL and the search scopes itself to that subreddit automatically.

**Skip Comments**... comments are collected by default, so flip this on if you only want post rows.

> **Watch out for this one.** If you sort a subreddit by `hot` and then set a short *Newer than* window, your run can come back empty. Hot posts are often days old, so the time filter eats all of them. Switch *Subreddit Sort Order* to `new` when you want recent content.

**When to end run** lets you choose between stopping once credits run out or once all tasks are consumed.

Now open *Advanced settings*.

![Reddit Scraper advanced settings including Slots](https://d37gzvgyugjozl.cloudfront.net/scrapereddit_step_3_adjust_behavior_ac06ef6e98.png "width=1348;height=1609")

**Max Results Per Task** caps rows per URL, **Max Unique Results** caps the whole run, and **Max Comment Depth** controls nesting.

**Search Within Comments** is the one I'd pay attention to. Flip it on and Reddit searches comment bodies instead of post titles.

**Slots** controls concurrency, up to 20 on this scraper, defaulting to 1.

👉 [How Slots work](https://help.lobstr.io/core-concepts/slots)

Next is the Notification tab, where you can get an email when a run finishes or errors out.

![Reddit Scraper notification settings](https://d37gzvgyugjozl.cloudfront.net/scrapereddit_step_3_adjust_behavior_2_fbdc7f2ffc.png "width=1656;height=624")

👉 [Notification settings](https://help.lobstr.io/core-concepts/notifications)

### Step 4: Launch

In the Launch tab you've got two options:

1. Manually
2. Repeatedly

**Manually means instant launch.** Leave it selected, click *Save & Extract*, confirm with *Yes*, and the scraper starts pulling data.

![Launching the Reddit Scraper manually or on a schedule](https://d37gzvgyugjozl.cloudfront.net/scrapereddit_step_4_launch_84b3bffc0d.png "width=1556;height=1122")

**But what if you want to track a subreddit over time?**

That's where *Repeatedly* comes in.

Set an interval in minutes, hours, days, weeks or months, pick a timezone and start time, then choose *Everyday* or *Every weekday*.

Perfect for watching a subreddit for brand mentions as they appear.

### Step 5: Enjoy

Once a run finishes, hit download and **export your results as CSV, Excel, JSON or JSONL**.

![Scraped Reddit posts in the lobstr.io results table](https://d37gzvgyugjozl.cloudfront.net/scrapereddit_step_5_enjoy_b4e4f7612f.jpg "width=1232;height=565")

Filter the `type` column to split posts, comments and subreddit rows.

👉 [How to download your results](https://help.lobstr.io/data/download-results)

But downloading a file manually every run? No thanks.

**lobstr.io automates delivery.** Click the Delivery icon and pick your method.

![lobstr.io delivery options: email, Google Sheets, SFTP, webhook and Amazon S3](https://d37gzvgyugjozl.cloudfront.net/scrapereddit_step_5_enjoy_2_30ca57a967.jpg "width=1232;height=784")

**Google Sheets is my personal favorite.** You can also push to Amazon S3, or just add an email address and get a CSV in your inbox after every run.

> Results stay downloadable for 45 days on Starter and Pro, 90 days on Team and Business and 7 days on the free plan, so set up automated delivery if you want to keep them longer.

So you've got the data. Now what?

## What can you do with scraped Reddit data

You can wire the whole thing into [Make](https://help.lobstr.io/integrations/make), [n8n](https://help.lobstr.io/integrations/n8n), or the [REST API](https://docs.lobstr.io) and never touch the dashboard again.

Or skip all of that and hand the scraper to your AI agent. lobstr.io's MCP server plugs into Claude, ChatGPT, Cursor or VS Code with one URL, and you sign in with your lobstr.io account, no API key.

If your client reads a config file, this is the whole setup:

```json
{
  "mcpServers": {
    "lobstr": {
      "url": "https://mcp.lobstr.io/mcp"
    }
  }
}
```

In Claude it's even simpler... *Settings → Connectors → Add custom connector*, paste the URL, done. Reading is free, and your agent asks before any run spends credits.

[Connect lobstr.io to your AI agent](https://mcp.lobstr.io/)

Here's what I'd actually do with it.

### 1. Pain point research

This is the one I use constantly, and I'll be honest with you... **I used this exact scraper to research the article you're reading right now.**

Not through the dashboard though. I drove it through lobstr.io's MCP server straight from my AI client, searching subreddits and reading results without leaving the conversation.

Turn on *Search Within Comments*, give it a keyword, and you get threads where people describe their problems in their own words. That's your content calendar and product roadmap in one export.

> If you want me to cover **how to use lobstr.io's Reddit MCP** for any of these workflows, [ping me on LinkedIn](https://pk.linkedin.com/in/shehriar-ahmad-awan).

### 2. Brand and competitor monitoring

Schedule a keyword scoped to the subreddits your customers live in, set Notification to fire on success, and pipe it to Sheets.

You'll spot a complaint thread before it reaches your support inbox.

### 3. Trend detection without the math

`score_per_hour`, `age_hours`, `comment_to_score_ratio` and `is_high_engagement` are calculated for every row.

You don't compute velocity from timestamps. **It's already in the export.**

### 4. Community research before you post

The subreddit row hands you `subreddit_rules` and `subreddit_resources` as structured data, plus `weekly_active_users`.

Read the room before you post into it.

### 5. Feeding an AI agent

No Reddit account means no login step to automate and nothing to keep alive.

Connect it to [Claude](https://help.lobstr.io/integrations/claude) or [ChatGPT](https://help.lobstr.io/integrations/chatgpt) and your agent runs the scraper itself.

Need more ideas? Here's a [Reddit topics scraper](https://www.lobstr.io/ideas/reddit-topics-scraper) use case to get you started.

## FAQs

### Is it legal to scrape Reddit

**Not a crime, but it does breach Reddit's User Agreement.** Section 7 bars collecting data "by any means (automated or otherwise)" without a separate agreement, and `robots.txt` now disallows every crawler.

After **Van Buren** (2021) and **hiQ v. LinkedIn** (2022), the criminal route for reading public, logged out pages is effectively closed. The live risk is civil, and Reddit is currently suing Anthropic on contract and unjust enrichment grounds, plus SerpApi and others on a DMCA anti-circumvention theory.

Risk concentrates on logging in, defeating controls and reselling data. Not on reading. I've covered this properly in [is data scraping illegal?](https://www.lobstr.io/blog/data-scraping-illegal) and across our [legal series](https://www.lobstr.io/blog/category/legal).

This isn't legal advice. Talk to a professional if compliance matters to you.

### Does the Reddit `.json` trick still work

**No, it returns a 403 to anonymous clients.** Reddit announced the shutdown on 28 May 2026 and breakage was reported publicly within a day. `api.reddit.com` is gone too, and `old.reddit.com` just redirects you to a login.

### Do I need a Reddit account or an API key

**Neither.** No login, no cookies, no browser extension, no OAuth app. I ran this scraper with no account attached and it worked fine.

### I scraped a subreddit but got no comments, why

**Comments only come from post URLs**, the ones containing `/comments/`. A subreddit URL returns its posts plus a single subreddit metadata row. Feed the post URLs in to collect comment rows.

### How much does it cost

**$2 per 1,000 results on Starter**, dropping to $0.50 per 1,000 at scale. One result is one credit, there are no add-ons to bill on top, and runs that return nothing cost nothing.

### How fast is it

**Up to 280 results per minute.** A run can use up to 20 Slots of concurrency, though the Slots field defaults to 1.

### Can I schedule it

**Yes, right from the Launch step.** Minutes, hours, days, weeks or months, with a timezone and start time. No manual run required first.

### Why not just use Reddit's official API

**Because it needs OAuth and a commercial agreement.** The Data API Terms require a separate agreement for commercial use, and the free tier caps at 100 queries per minute. Reddit is also winding the Public Data API down in favour of Devvit.

## Conclusion

That's a wrap on **how to scrape Reddit** posts, comments and subreddits, without coding and without ever logging in.

The `.json` era is over. But the data is still public, still readable, and still the most honest research material on the internet.

If you want me to cover a related topic, or walk through driving this scraper entirely through MCP, [ping me on LinkedIn](https://pk.linkedin.com/in/shehriar-ahmad-awan). 🦞

---

## Scrapers in this article

- [Reddit Scraper](https://www.lobstr.io/store/reddit-scraper): Extract 64 data points from any Reddit subreddit, post, user or search — posts, comments, authors & subreddit metadata all in one run. (1 credit per result)

## Related articles

- [Best Reddit Scrapers 2026 [No-Code + API Edition]](https://www.lobstr.io/blog/best-reddit-scrapers)

Tags: Tutorials, No Code, Web Scraping, Social Media

_Canonical HTML: https://www.lobstr.io/blog/how-to-scrape-reddit_

More for agents: https://www.lobstr.io/llms.txt, pricing in Markdown: https://www.lobstr.io/pricing.md