← Back

Two Berlin Newspaper Scrapers

What Is It?

Two small Telegram bots from late 2020, built to skill up my German: one watches Berliner Morgenpost, the other Berliner Zeitung, each posting new articles to a Telegram channel as they appear. Reading local news in German, delivered automatically, seemed like a better way to practice than flashcards — and it doubled as an excuse to get deeper into Python 3.

Two versions, three weeks apart. morgenpost_unofficial came first: it scrapes the Morgenpost sidebar every 12 hours and posts whatever's there — simple, no memory of what it already posted. berliner_zeitung_official came a few weeks later and is the more grown-up version: it checks a MySQL table for each article's link before posting, so nothing gets sent twice, runs every 4 hours instead of 12, and pulls out the title, lead paragraph, and hashtag instead of just a title and link. It also picked up a proper deployment pipeline — Jenkins building a Docker image, pushing it to ECR, and rolling it out on OpenShift — where the first one didn't have any of that.

The Stack

Built With

What Came Next

The Python 3 practice from these two didn't stay a hobby thing for long — the scraping patterns here (BeautifulSoup over a fetched page, structured extraction, idempotent re-runs) are the same ones behind the freelance project I took on right after: an annual report finder for Chinese stocks.

This was also right around the time Python 2 was still hanging around in production code alongside Python 3, which is what prompted the earlier piece on how modules work in both versions — both were genuinely in demand at the time, so it was worth knowing the differences.