3 Ideas to Generate Passive Income with a Web Scraping Tool and Some Elbow Grease
Do you know the proverb “All days are equal, but they’re not the same”? I’ve heard it recently, and I’ve come to realize that it applies to a lot more than just days. For example — web scraping projects.
Ok, that’s a bit of a stretch, but let me explain. I’ve covered quite a few examples of web scraping use cases. I can say that they’ve been equally fun, for the most part. But some projects are more equal than others, know what I mean?
I wanted to dedicate this article to those kinds of web scraping use cases, namely the ones having to do with automation. Gathering data for a report is cool and all, but the idea of making a script that works tirelessly while I’m doing something else is more appealing to me.
The kicker for me and possibly for you too is the opportunity to generate passive income, so I’ll be going over monetization options as well. Well then, here are a few web use cases where you wind up the web scraping bot and then just let it do its thing:
Comparison Websites are always right
When I’m looking to buy something, comparison websites are my lord and savior. As soon as I’m done shopping, though, I immediately forget they exist.
The admins for comparison websites don’t seem to mind, though. After all, they built a data collection system, designed a decent-looking website, and went on to the next project.
The data extraction process is the key component of any comparison website. They’re basically offering structured data to visitors, so that information must be accurate and up to date. How do you do that? With web scraping software, of course.
To choose a data extraction tool, ask yourself the following questions:
- How many requests will it have to do? If the answer is many, you’ll need something to handle plenty of concurrent requests and have an advantageous cost-per-request. I’d recommend an API.
- Will it scrape plenty of different pages or mostly revisit the same ones? The more unique pages you’ll scrape, the more code you have to prepare in advance.
- How fast is the data ready? Comparison websites have to have all the latest data as soon as possible; it’s what makes them valuable. If you’re just starting out, make sure you can get your database sorted out in good time. Otherwise, the project is dead in the water.
Here’s an idea on how to get an extra edge on other comparison websites: while product specs stay the same, prices can change at a moment’s notice. Set a short timer for the scraper to check only the costs for the products you’re displaying on the comparison site. You can even code it in such a way that when a visitor wants data on a specific product, the scraper grabs its price at that very moment, ensuring 100% data validity.
How to monetize it
There are two ways, and they’re not mutually exclusive. First, you can use affiliate links for the products you show. If someone follows your link and buys the product, you get a cut. Second, you can just run ads on the site. Once you get traction and a decent number of visitors, it shouldn’t be too difficult to find someone interested in running ads on the site.
From day trading to always trading
Want a project that you can sink months or even years into and still feel like it has room for improvement? I probably split the room with that question. Anyway, here’s automated stock trading.
The basic principle is super simple: you use a web scraping bot to gather relevant data. The very first thing would be stock prices and their evolution. Web scraping isn’t your only option here, but it gives a good deal of control, and you’ll likely need it for other things too.
The web scraper is there to gather additional information: news, opinions for sentiment analysis, Trump’s tweets, whatever you want. It’s the stock market, no one really knows what’s going on.
Back on track, you scrape whatever you’d consider valuable data. Stock prices (or even worse, cryptocurrency) can be volatile, so make sure the scraper works around the clock, bringing in fresh data.
Then you define the logic for your automated trader. This is the part that can take forever. You could tell it to buy low and sell high and call it a day, but if it was that easy, we’d all be rich. Or at least I’d be.
Anyway, the idea is that humans look at outside input, the information around them, and act on it. Bots can do that too, and the stock market is an excellent place to try since it’s a world dominated by numbers, statistics, and tonnes of processed data.
How to monetize it
This one should be pretty easy to figure out, no? Make the bot, give it some dough, hope it makes more, tweak endlessly. Please don’t invest more money than you’re willing to lose. I don’t want you on my conscience.
Buy, sell, profit
I’ll say this at the very start: scalpers suck, and please don’t become one. Now, let’s see how you can buy discounted products, resell them at a higher price, and pocket the difference.
Just about anything goes on sale from time to time. For most people, that means that there’s always a chance they’ll get a better deal on something they want/need. For you, though, every sale can be an opportunity to make a bit of cash. Here’s the step-by-step:
- Set up a web scraper to periodically check online shops for significant discounts (ex: 50% off). You can do this for just about any product but keep in mind that the idea is to sell it fast.
- Use a script that buys those heavily discounted goods whenever the opportunity arises.
- Wait for the special offer to expire.
- Resell the product for less than you bought it but more than its regular price.
Reselling shouldn’t be challenging when you’re offering a brand new product for less than the online store where you originally purchased it.
Nothing is stopping you from doing this manually. Well, it’d be very dull and not that lucrative, but you can do it if you want. Bots are a much better option because, again, it’s a mostly passive source of income.
How to monetize it
As long as you get a bot to do steps one and two (three happens on its own), you just have to spend one or two hours every week listing your purchases for sale. Every sold article nets you a percentage of its value as profit. For example, you buy a phone at 50% off, sell it at a 25% discount and keep 25% for yourself. It may not sound like much but do that four times, and you’ve just made enough money to buy a new phone (at full price).
That’s a bit of a simplification, but depending on how vigilant your scraper is and how easy it is to resell the products, you might just find a nice way to supplement your income.
This kind of activity is often viewed negatively because of scalpers, and it’s partially deserved. Buying in bulk to create scarcity and gouge prices is scummy. I’m perpetually mad about GPU prices.
Making a few bucks on a discount while also selling below market price is a different matter.
How do I get in on these projects?
Honestly, you don’t need much. The only things I’d say you can’t go without are time and coding experience, which comes naturally when you invest time. That being said, these projects can lift off the ground pretty fast if you use the right tools instead of building everything from scratch.
The common element here was web scraping, and on that front, I can point you in the right direction. Here’s what I think about a bunch of different data extraction tools, frameworks, and APIs. Which one to choose and how to use it, that’s up to you.
To my mind, any project that can be brought to MVP level within two weeks of coding on your off hours is worth trying. If the program leads nowhere, you learn a valuable lesson. If it actually works, gains traction, and maybe even turns a profit, then you learn a lesson and get paid for it.
Good luck!
