# How to Scrape Twitter Search Results without coding [2024 Edition]

> In this article, we’ll learn how to scrape all tweets from X (formally Twitter) search results without any API in 2024. No coding required. Totally for free.

- **Author**: Shehriar Awan
- **Published**: 2024-03-15 (15 Mar 2024)
- **Reading time**: 9 min
- **Canonical**: https://www.lobstr.io/blog/scrape-twitter-search-results

---

Want to **analyze Twitter trends without complex code**? Twitter (now X) is a goldmine of opinions and real-time reactions, but accessing that data often feels out of reach for non-coders. 

![scrape twitter search results - image10.png](https://d37gzvgyugjozl.cloudfront.net/scrape_twitter_search_results_image10_df05595bd2.png "width=927;height=919")

[Scrape Twitter Search results for free](https://www.lobstr.io/store/twitter-search-results-scraper)

In this article, we’re going to learn **how to scrape Twitter search results** i.e. top and latest tweets from any search query, hashtag, or trend – without coding and absolutely for free.

But why would anyone want to scrape X trends data? Let’s explore some use cases.

## Why scrape Twitter search results?

With over [420 million monthly active users](https://www.businessofapps.com/data/twitter-statistics/), Twitter is one of the most engaging social media platforms. Extracting Twitter data can help:

![twitter similarweb - image2.png](https://d37gzvgyugjozl.cloudfront.net/twitter_similarweb_image2_d641ae9eef.png "width=1507;height=645")

* **Marketers**: Track brand mentions, campaign performance, and analyze sentiment around your products or services.
* **Researchers**: Study public opinion, track social movements, and analyze language trends on Twitter.
* **Businesses**: Identify potential leads, collect customer feedback, and improve overall customer service.
* **Competitive Analysis**: Monitor what people are saying about your competitors' products, services, and overall brand image.
* **Training Machine Learning Models:** Train various ML models like sentiment analysis classifiers, topic modeling algorithms etc

But is it legal to scrape Twitter search data?

## Is it legal to scrape Twitter search results?

**Web scraping is legal until you’re only scraping publicly available data** and no personal information is being collected. 

At least that’s [what people believe](https://www.reddit.com/r/webscraping/comments/nb3a7t/is_scraping_twitter_commercially_legal/):

![is scraping legal - image22.png](https://d37gzvgyugjozl.cloudfront.net/is_scraping_legal_image22_ee61942e12.png "width=973;height=675")

But is it true? Well, we can’t decide based on a random reddit opinion. So upon further research, I found **2 most relevant and recent law-suites about legality of scraping**.

* Meta vs BrandTotal
* LinkedIn vs HiQ Labs

In the [Meta vs BrandTotal](https://www.govinfo.gov/app/details/USCOURTS-cand-3_20-cv-07182/context) case, Meta alleged that Israeli ad firm BrandTotal was [collecting user information without permission](https://www.courthousenews.com/judge-rules-brandtotals-data-harvesting-violates-metas-terms-of-use-and-anti-hacking-laws/).

![meta vs brandtotal - image1.png](https://d37gzvgyugjozl.cloudfront.net/meta_vs_brandtotal_image1_03e2cd7f52.png "width=820;height=343")

A federal judge ruled that BrandTotal violated Facebook's terms of use and anti-hacking laws.

Later BrandTotal updated its product, limiting itself to **only public data collection**; in return, Facebook agreed to cease action against the new version.

In a similar case – the famous [LinkedIn vs HiQ Labs](https://cdn.ca9.uscourts.gov/datastore/opinions/2022/04/18/17-16783.pdf) lawsuit, Linkedin sued HiQ Labs for scraping public user profiles. But **the court ruled in favor of HiQ Labs**.

![linkedin vs hiq - image20.png](https://d37gzvgyugjozl.cloudfront.net/linkedin_vs_hiq_image20_c8ece25a9f.png "width=957;height=364")

**After the LinkedIn vs HiQ Labs case verdict, it has been solidified that scraping publicly available data is fully legal**.

Since X search results are public tweets, they don’t have any copyright, and they don’t disclose any personal identifiable information, it’s fully legal to scrape them.

For a detailed understanding, we have a dedicated list of articles about [web scraping legality](https://www.lobstr.io/blog/category/legal).

But in the same reddit post I mentioned above, [Mr. TheElectricSlide2](https://www.reddit.com/r/webscraping/comments/nb3a7t/comment/gxxh38m/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button) suggests using the official X API. Is he right about the official API?

![use twitter api reddit comment - image21.png](https://d37gzvgyugjozl.cloudfront.net/use_twitter_api_reddit_comment_image21_00334e0671.png "width=577;height=144")

## Why not use the official Twitter API?

X (formerly Twitter) does offer an **API for collecting and analyzing data**. Then why don’t we use it? There are plenty of reasons for not doing so.

* It’s too expensive
* It has limitations
* You need coding skills

I’ve already discussed all these reasons in detail in my [how to scrape user tweets article](https://www.lobstr.io/blog/scrape-tweets-without-coding). Do check it out for a better understanding.

For a step by step tutorial on how to extract data from Twitter using python, check out this article: **[How to scrape Tweets with Python and requests in 2023?](https://www.lobstr.io/blog/how-scraping-tweets-with-python-and-requests-in-2023)**

Now let’s address the elephant in the room – how to scrape Twitter search results without any API and without coding a custom scraper?


## How to scrape Twitter search results without coding?

The best way to scrape Twitter data is **using no-code tools**. That’s what we’re going to do. We’ll be using [Twitter Search Results Scraper](https://www.lobstr.io/store/twitter-search-results-scraper) – the best no-code Twitter scraper in the market.

![twitter search results scraper - image13.png](https://d37gzvgyugjozl.cloudfront.net/1_Akiax1w_N5_cbfb9acc56.png "width=955;height=610")

[Scrape Twitter Search Results for free](https://www.lobstr.io/store/twitter-search-results-scraper)

### Cool features

* Scrape all latest and top tweets
* 25 vital data attributes e.g. tweet data, user data
* 125+ tweets per minute speed
* Cloud-based, no install required
* Schedule to scrape repeatedly and monitor
* Export data directly to Google Sheets and Amazon S3
* Developer-friendly API access

### Pricing

![lobstr pricing - image19.png](https://d37gzvgyugjozl.cloudfront.net/e_Ry4w_Ce_N_Mv_e6e7199149.png "width=1200;height=705")

* Scrape 100 tweets per month for free
* On paid plans, __cost per 1000 tweets starts at $2__
* At scale, cost drops to __$0.5 per 1000 tweets__

Now let’s get started ✨

## Step by step guide to scraping Twitter search results using Lobstr.io

We’re going to scrape tweets from a Twitter trend published during a certain date range. We’ll do this in **6 really simple steps**.

1. Get Twitter search URL
2. Create squid and Sync account
3. Add tasks
4. Adjust behavior
5. Launch
6. Enjoy

Let’s go!!! 💨

### Step 1 - Get Twitter search URL

First step is to get the search URL from Twitter. 

With Lobstr, you can scrape both top and latest tweets from any trend or search query by simply **copying and pasting the search URL**.

We’re going to scrape all top tweets on #bitcoin posted between _March 9 - March 13, 2024_. Let’s use Twitter advanced search to add date range.

![twitter advanced search - image18.png](https://d37gzvgyugjozl.cloudfront.net/twitter_advanced_search_image18_2de60bd0e2.png "width=1069;height=453")

Here's the URL:
https://twitter.com/search?q=(%23bitcoin)%20until%3A2024-03-13%20since%3A2024-03-09&src=typed_query

Now simply copy the URL. If you want to scrape the latest tweets, move to the _Latest_ tab and then copy the URL. It’ll have a `f=live` parameter. 

![get twitter search url - image4.png](https://d37gzvgyugjozl.cloudfront.net/get_twitter_search_url_image4_45713e0ff3.png "width=1007;height=431")

Now let’s move to the next step.

### Step 2 - Create Squid and sync Twitter account

Next, go to your lobstr.io dashboard. Don’t have an account yet? It’s free! Go create one first. Once you’re in, click the _new squid_ button and search ‘Twitter’.

![create squid - image6.png](https://d37gzvgyugjozl.cloudfront.net/create_squid_image6_f2df8d4ab4.png "width=625;height=606")

Select **Twitter Search Results Scraper** and you’ll see a new pop-up window asking you to sync a Twitter account.

![sync twitter account - image17.png](https://d37gzvgyugjozl.cloudfront.net/sync_twitter_account_image17_eaea87e034.png "width=628;height=452")


> Last year, X decided to hide tweets behind login [to prevent AI data scraping](https://www.businessinsider.com/elon-musk-twitter-users-being-forced-sign-in-see-tweets-2023-7). You can access tweets data only if you’re logged in. That’s why we need you to sync your account.

To sync your Twitter account, all you have to do is install the [Lobstr account sync Chrome extension](https://chromewebstore.google.com/detail/lobstrio-account-sync/oelngbppodofbbffdicjmmmnhgcchokd). No login credentials needed, totally safe.

![lobstr account sync addon - image15.png](https://d37gzvgyugjozl.cloudfront.net/lobstr_account_sync_addon_image15_d0d0e84f57.png "width=1425;height=799")

Once the Chrome Addon is installed, click _Yes I want to sync_, and you’re good to go.

### Step 3 - Add tasks

This step is easy peasy. Just **paste the Twitter search URL** you copied in step 1 and click _Add+_. That’s it. But what if I’ve hundreds of URLs?

![add url to tasks - image11.png](https://d37gzvgyugjozl.cloudfront.net/add_url_to_tasks_image11_c6ffc56d24.png "width=941;height=713")

You can add as many tasks as you please, but doing it manually is time consuming. For that, you can use the _upload file_ option and [upload all tasks in bulk](https://lobstr.crisp.help/en/article/how-to-upload-a-list-of-tasks-iv9pu0/) in a single click.

After adding tasks, click _Save_ and you’ll see the settings menu.

### Step 4 - Adjust behavior

In basic settings, you can **choose the number of tweets to scrape** per task. Want to scrape all tweets? Just leave it blank. I need 250 tweets only, so let’s set _Max Results_ to 250.

![set max results - image5.png](https://d37gzvgyugjozl.cloudfront.net/set_max_results_image5_d327702425.png "width=961;height=701")

When to end run option will help you configure the freshness and quantity of data. If you want to **collect fresh tweets** every time the scraper runs, select the first option.

If you need **all tweets available** on a search query/trend, select the second option. It’s best for scraping tweets posted in specific date ranges.

![set concurrency - image7.png](https://d37gzvgyugjozl.cloudfront.net/set_concurrency_image7_ad0e8eedb4.png "width=815;height=355")

In advanced settings, you can give your crawler super speed. Use **concurrency** to increase the number of bots deployed per job.

_more concurrency (more no. of bots) = more speed_

You can remove duplicate results by toggling _Unique Results_, and for a better output in Excel, remove the line breaks from text using the _No Line Breaks_ option.

After adjusting the crawler's behavior, click _Save_ to move to notifications.

![set notifications - image14.png](https://d37gzvgyugjozl.cloudfront.net/set_notifications_image14_90fbfc3d9a.png "width=963;height=395")

You can opt to **receive email notifications** when a run completes successfully or stops due to any issue.

Now we’re ready to launch 🚀

### Step 5 - Launch

For instant data collection, you can **launch the scraper manually**. Just click the _Save & Extract_ button and your data collection will begin.

![launch manually - image9.png](https://d37gzvgyugjozl.cloudfront.net/launch_manually_image9_8893527b28.png "width=977;height=762")

But what if I want to **monitor a trend continuously and collect fresh tweets** every week or every day? That’s where the schedule feature comes into play. 

You can schedule the Twitter search results scraper to run **automatically and repeatedly** on time and date of your choice. Whether it’s hourly, daily, weekly, or monthly.

![schedule launch - image16.png](https://d37gzvgyugjozl.cloudfront.net/schedule_launch_image16_ef92a45432.png "width=845;height=601")

For example, I can schedule this crawler to collect the latest tweets every 2 hours, starting at 10 AM.

After setting your launch preferences, click _Save_ and your launch sequence is complete. ✨

![crawler console - image12.png](https://d37gzvgyugjozl.cloudfront.net/crawler_console_image12_40292f6d68.png "width=1872;height=911")

### Step 6 - Enjoy

And here we go, in less than 2 minutes, we’ve collected 250 top tweets with 25 data attributes, from #bitcoin, filtered by date range.

![download results - image8.png](https://d37gzvgyugjozl.cloudfront.net/download_results_image8_18c6ae7ec4.png "width=1838;height=661")

You can view the results in the dashboard or download them as a csv file. Comfortable with Excel? Here’s how to convert [csv to Excel](https://lobstr.crisp.help/en/article/how-to-convert-a-csv-file-to-excel-kw9j8l/).

![results csv file - image3.png](https://d37gzvgyugjozl.cloudfront.net/results_csv_file_image3_56d855e528.png "width=1920;height=857")

But what if I want to export them directly to Google Sheet? 

You can use the _Delivery_ button  to configure your crawler to[ directly send the collected data to a Google Sheet](https://lobstr.crisp.help/en/article/how-to-export-data-to-googlesheet-q9onsm/) or Amazon S3.

That’s it. We just extracted 250 tweets from a Twitter trend in less than 2 minutes. Now let me answer some frequently asked questions for you.

## FAQs

### Does Twitter Search Results Scraper also offer data in JSON format?

**No, you can only download data as csv** or export it to any of the delivery options available. But you can easily convert CSV to JSON [using this tool](https://csvjson.com/).

### How many tweets can you scrape?

**There’s no rate limit at crawler level**. You can scrape all tweets available on a Twitter trend or search query using this automation.

### How to collect all tweets from a Twitter profile?

You can use [Twitter User Tweets Scraper](https://www.lobstr.io/store/twitter-profile-scraper) to scrape all tweets from any public Twitter account.

### How to scrape Twitter profile data?

Use our no-code [Twitter Profile Scraper](https://www.lobstr.io/store/twitter-profile-scraper) for collecting all vital data about public Twitter profiles.

### How can I scrape data from Twitter using Python?

You can **use python libraries** like [tweepy](https://www.tweepy.org/) for web scraping Twitter that utilizes X API, or try scraping tools like [snscrape](https://github.com/JustAnotherArchivist/snscrape) and [twint](https://github.com/twintproject/twint). But they’re complex, and have limitations.

For collecting twitter data at scale, you can also use [Lobstr’s API](https://lobstrio.docs.apiary.io/) with python or any other language of your choice.

## Conclusion

That’s a wrap on scraping Twitter search results without coding. Try the Twitter Search Results Scraper – it’s free forever.

Check out Lobstr’s store for more amazing web scraping tools. Didn’t find what you were looking for? [Submit your idea here](https://www.lobstr.io/ideas).

---

## Scrapers in this article

- [Twitter Search Results Scraper](https://www.lobstr.io/store/twitter-search-results-scraper): Scrape all tweets data from a Twitter Search URL and export it to googlesheet. Retweets and quotes included (1 credit per result)
- [Twitter User Tweets Scraper](https://www.lobstr.io/store/twitter-user-tweets-scraper): Extract all tweets from any X/Twitter user — 39 attributes per tweet, no code needed. (1 credit per result)
- [Twitter Profile Scraper](https://www.lobstr.io/store/twitter-profile-scraper): Scrape unlimited Twitter profiles with 20+ data attributes. (1 credit per result)

## Related articles

- [How to Scrape Tweets from Twitter (No Code) - 2026 Edition](https://www.lobstr.io/blog/scrape-tweets-without-coding)
- [How to scrape Tweets with Python and requests in 2023?](https://www.lobstr.io/blog/how-scraping-tweets-with-python-and-requests-in-2023)

Tags: Twitter, Web Scraping, Tutorials, Use Cases, No Code

_Canonical HTML: https://www.lobstr.io/blog/scrape-twitter-search-results_

More for agents: https://www.lobstr.io/llms.txt, pricing in Markdown: https://www.lobstr.io/pricing.md