Do you know the proverb “All days are equal, but they’re not the same”? I’ve heard it recently, and I’ve come to realize that it applies to a lot more than just days. For example — web scraping projects.

Ok, that’s a bit of a stretch, but let me explain. I’ve covered quite a few examples of web scraping use cases. I can say that they’ve been equally fun, for the most part. But some projects are more equal than others, know what I mean?

I wanted to dedicate this article to those kinds of web scraping use cases, namely…

In an increasingly complex world with equally tangled problems, a simple solution is a good solution. I’ll give you an example:

I want to listen to (not read) articles from the internet while I’m out in the sun. They’re freely available, but how do I turn them into audio files to enjoy on a mobile device?

The first and quickest solution is to find a product that solves this very problem for me. Audible comes to mind, but I want blog posts, not books, and I like the option of choosing any written content.

Well, I can probably build a…

Web scraping is a method of extracting vast volumes of data from websites and saving it to a local file on your device or to a database in spreadsheet form.

Many websites’ data can only be accessed by a web browser. They don’t allow you to keep a copy of this information for personal use. The only other choice is to manually copy and paste the results, which is a time-consuming task that can take hours or even days to complete.

Web scraping is a method of automating this operation so that, instead of manually downloading data from websites, the…

The Internet includes vast amounts of information, and having access to valuable data can really jumpstart a business. But manually collecting and storing anything we find helpful would be a pain in the neck.

After figuring out what data to gather, the other big issue is collecting it and storing it in a fast, straightforward manner. If you’re looking for a tool to go through thousands of web pages, look no further than a web scraping API.

An efficient one can provide you with all the data you need. …

At first glance, many might think that communication between computers or programs is simple. “They just talk in 1s and 0s” right? Well, not quite. That’d be like saying that humans communicate by making noises with their mouths. You’d be ignoring the fact that one set of noises is the English language, while another is French, and the particularly screamy noises are German and so on.

Point is, even computer programs can use different languages for communication. Application programming interfaces are meant to bridge the gap between various programs, to ensure that the data crosses the language barrier.

For people…

In our increasingly digital world, more and more business happens online. A company’s website is sometimes the primary way through which consumers can interact with their brand or product. This is especially true of news companies, which have moved away from print media and embraced the online world.

With all that information on the Internet, a business can benefit from gathering targeted news data to grow and optimize its products and services. People consume more and more content nowadays. …

Data science is expanding the world’s ability to detect patterns, forecast the future, and draw profound knowledge from vast data sets as never before. Information is, without a doubt, the lifeblood of every data science project. Web scraping makes sense in fueling data science use cases because the web is now the most valuable source of data that has ever existed.

Web scraping is a technique for extracting vast volumes of data from websites for a range of purposes, including market analysis, machine learning algorithm enrichment, financial data aggregation, customer feedback monitoring, news tracking, and so on.

Browsers show information…

Being one of the largest business catalogs on the Internet, Yelp offers a lot of information about your competition and how people see your business. If you’ve never heard of Yelp, just know that you can find a lot of information about restaurants, hotels, dentists, gyms, landscapers, even plumbers on the platform. It’s a service-driven amusement park.

The benefits of web scraping on Yelp
Web scraping in action
1. Find the information you need
2. Inspect the source code
3. Setting up the project
4. Making the request
5. …

I’ve been writing about web scraping for a while now, both on general subjects and niche use cases or projects I found interesting. I like to think that I did a good job, but I’ve realized of late that before you embark on your next web scraping adventure, you need to explore some essential details.

I’ve compiled a list of frequently asked questions and did my best to come up with answers for them. …

Hello and welcome to today’s episode of “What makes an API tick?” where we will be talking about what makes an API shine and what are their most commonly encountered design mistakes.

When using or building an API, developers must ask themselves a few questions first. Is it consistent? Will it share too many things? Is the error handling done properly? If you want to know the answer to these questions, come along and find out for yourself!

Why APIs are important
Common API design mistakes you need to avoid
Lack of Data Validation
Data Oversharing
Unclear Error Codes

Dan Suciu

CEO & Co-Founder @Knoxon, Full Stack Developer

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store