I’ve set some resolutions at the beginning of the year: get vaccinated, resume travelling, start a passive revenue stream. The first two resolutions are works in progress. The last one was ignored up until now.

When you want to start a passive revenue stream, the consensus is to focus on something that you enjoy: I enjoy video games.

So, I want to start an Etsy print-on-demand business with gaming t-shirts.

The benefits of web scraping on Etsy

For those that live under a rock, Etsy is a community-driven global online marketplace where people can make, sell, and buy unique items.

The Etsy website gives three valuable pieces…

As a programmer, you’re using APIs on a daily basis. The role of an API is to make your life easier, reduce the number of headaches, and save you a lot of time.

Usually, in a web project, you use Payment, Newsletter, Social, and Map APIs. But there’s an unwritten rule for APIs: if you think about it, there’s an API for it.

Are you tired of hearing the horoscope from the radio or from the TV? You can connect to an API and get it from there.

Did you just hack your car’s dashboard and want to add the…

Creating your web scraper can be quite an easy task if you know what tools to use. Python, for example, is one of the most popular programming languages used to extract data from the web.

In one of my previous articles, I shared with you how to create your own Python web scraper in just a few steps, but you can use other libraries in order to get your job done too.

Now we’ll see which of these packages can help us and what their advantages are in building our web scraper. …

You might have heard about web scraping and sneakers. You probably own some sneakers but don’t know as much about web scraping.

One of the ways people like to use web scraping is to get the best prices on rare and expensive sneakers. It just so happens that these kinds of projects are also a great way to learn more about web scraping as a whole and what it can achieve.

In this article, I want to combine a web scraping tool and an auction site to show you how web scraping can be used. …

The popularity of web scraping is evolving nowadays at such an accelerated rate that it would be almost impossible not to get crossed responses when asking the big question: Is it legal? 👀

If you are exploring the Internet to find a legitimate answer that best fits your needs, you’ve come to the right place. This article aims to outline the legal concerns you must be aware of when scraping and also offer insights on how to minimize risks.

Spoiler alert: The question of whether or not web scraping is legal has no definitive and single response. …

The majority of what works now will soon become outdated. That applies to just about anything in the software world. In this case, depending on an out-of-date method of scraping SERP data would leave you stranded in the trenches.

Since Google and other search engines’ SERP structure is constantly changing, you must take advantage of a future-proof solution.

In this article, I’ll present such a solution, namely web scraping. Additionally, we’ll go over the reasons why you’ll want to use it.

What are keywords?

People often use keywords or key phrases to find helpful information in search engines like Google. …

Security is one of the most essential and foremost concerns of any API. The software security market is overgrowing year upon year — with the total estimated value reaching $153.16 billion as of 2020.

Whether you’re just beginning to develop an API or are looking to improve your existing APIs, there is a lot you can do to ensure that your software is secure and protected from cyber-attacks. API security threats should be considered in the design process from the very beginning to avoid possible attack vectors later on in the development cycle.

The world’s most skilled hackers are constantly…

Real estate changed at once with the upswing of the internet, as well as many other businesses that initially required oodles of bureaucracy. Technology now allows realty monopolies like CBRE to scrape and manage their listings worldwide.

Automation has seeped into fields like email marketing and customer support, so why not optimise property listings online to attract a larger pool of buyers?

Google Ads and many more eCommerce platforms have suggested that almost half of the buyer inquiries happen digitally. …

Web scraping is a well-established and mature technology used by thousands of programmers every single day around the world. You get to spend many hours creating your own web crawler and a lot of work optimising the code. But there’s still something missing.

Either you get blocked by the website, the HTML document is not complete, or you wait for decades until you even see some HTML. There is a lot of grunt work involved in making your web crawler genuinely effective.

Web scraping is no longer new, but there are still many tricks to be learned, and things that…

If you have the time, stop and think for a minute about what the software landscape would look like without APIs.

Pretty hard to imagine, right? I’ve been thinking about this very topic for a while and concluded that APIs are among the cornerstones of the interconnected world we live in. But how did they achieve that? How can APIs act as the bridge to so many different software products and not become impossibly complex?

The short answer is that it uses strict protocols, conventions that dictate how the programs interact with the API.

The long answer is this whole…

Dan Suciu

CEO & Co-Founder @Knoxon, Full Stack Developer

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store