📰 Project Showcase: Metis

4/5/2024 at 9:49:20 AM

Here we will go through the project showcase of Metis, a news aggregation platform which I have been developing in one form or another since 2020.

The Origins

The project actually originated as a spin-off project from another project I was working on, called Plutus, which is a market data and trading platform.

You can find the predecessor for it right here: GitHub

The Features

  • Around 2000 news sources (RSS feed, websites, archives and newsletters) collecting news articles in real-time, from around 250 publishers
  • 2M+ entries collected in the past 8 months
  • Collecting market data from Alpaca and Binance
  • Topics, topic relations and topic classification
  • Around 13k assets and 15k markets

The Stack

  • Typescript
  • Frontend: React with React Router, TailwindCSS, Jest, Cypress
  • Backend: NestJS, TypeORM, BullMQ
  • Databases: PostgreSQL, Redis, ClickHouse, Meilisearch
  • Storage: S3/Minio
  • Queue: RabbitMQ
  • Tools: NX, Yarn, Tsup
  • CI/CD: GitHub Actions, Docker, Docker Swarm
  • Monitoring: Grafana, Prometheus, Loki

The Result