📰 Project Showcase: Metis
4/5/2024 at 9:49:20 AM
Here we will go through the project showcase of Metis, a news aggregation platform which I have been developing in one form or another since 2020.
The Origins
The project actually originated as a spin-off project from another project I was working on, called Plutus, which is a market data and trading platform.
You can find the predecessor for it right here: GitHub
The Features
- Around 2000 news sources (RSS feed, websites, archives and newsletters) collecting news articles in real-time, from around 250 publishers
- 2M+ entries collected in the past 8 months
- Collecting market data from Alpaca and Binance
- Topics, topic relations and topic classification
- Around 13k assets and 15k markets
The Stack
- Typescript
- Frontend: React with React Router, TailwindCSS, Jest, Cypress
- Backend: NestJS, TypeORM, BullMQ
- Databases: PostgreSQL, Redis, ClickHouse, Meilisearch
- Storage: S3/Minio
- Queue: RabbitMQ
- Tools: NX, Yarn, Tsup
- CI/CD: GitHub Actions, Docker, Docker Swarm
- Monitoring: Grafana, Prometheus, Loki
