Skip to content

iiiyu/marketstream

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

6 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

MarketStream

πŸš€ Free Self-Hosted Real-Time Stock Data Platform

A robust HTTP API service that provides real-time market data and historical candle data using the TradingView WebSocket Client library. MarketStream gives you unlimited access to TradingView's data through your own self-hosted platform.

πŸš€ Features

Core Capabilities

  • REST API for real-time market data access
  • Real-time quote data with comprehensive market information
  • Historical candle data with multiple timeframes
  • Dynamic symbol management (add/remove subscriptions during runtime)
  • PostgreSQL integration for persistent data storage
  • High-performance caching with Ristretto for real-time quotes
  • Automatic database migrations with Ent ORM

Enhanced Architecture

  • Clean interfaces for better testability and dependency injection
  • Message router pattern for extensible WebSocket message processing
  • Structured error handling with context and retry logic
  • Comprehensive input validation for security and data integrity
  • Advanced configuration system with YAML, environment variables, and CLI flags
  • Development tooling with Makefile automation

Technical Indicators Framework

  • Comprehensive technical analysis supporting 10+ indicators
  • Study session management for grouping related indicators
  • Real-time calculation with candle data integration
  • REST API endpoints for indicator CRUD operations

πŸ“¦ Installation

go get github.com/iiiyu/marketstream

⚑ Quick Start

1. Setup Development Environment

# Clone the repository
git clone https://github.com/iiiyu/marketstream
cd marketstream

# Install development tools and dependencies
make setup

# Copy example configuration
cp config.example.yaml config.yaml

2. Configure the Application

Edit config.yaml with your settings:

server:
  port: "3333"
  environment: "development"

database:
  host: "localhost"
  port: 5432
  user: "postgres"
  password: "your_password"
  db_name: "tradingview"
  ssl_mode: "disable"

tradingview:
  device_token: "your_device_token_here"
  session_id: "your_session_id_here" 
  session_sign: "your_session_signature_here"

logging:
  level: "info"
  format: "json"

Or use environment variables:

# Database Configuration
DB_HOST=localhost
DB_PORT=5432
DB_USER=postgres
DB_PASSWORD=your_password
DB_NAME=tradingview
DB_SSLMODE=disable

# TradingView Configuration (Required)
TRADINGVIEW_DEVICE_TOKEN=your_device_token
TRADINGVIEW_SESSION_ID=your_session_id
TRADINGVIEW_SESSION_SIGN=your_session_signature

# API Authentication (Required for production)
API_KEY=your_secure_api_key_here
AUTH_ENABLED=true

# Optional Server Configuration
PORT=3333
LOG_LEVEL=info

3. Run the Application

# Development mode with live reload
make dev

# Or run directly
make run

# Or build and run
make build
./bin/tradingview-http-service

πŸ” API Authentication

MarketStream uses API key authentication to secure all endpoints except /health.

Configuration

Set your API key in one of the following ways:

Environment Variables:

API_KEY=your_secure_api_key_here
AUTH_ENABLED=true

Configuration File (config.yaml):

auth:
  enabled: true
  api_key: "your_secure_api_key_here"

Command Line:

./marketstream --auth.enabled=true --auth.api_key=your_key

Making Authenticated Requests

Include your API key in requests using one of these methods:

Option 1: X-API-Key Header

curl -H "X-API-Key: your_api_key_here" http://localhost:3333/symbols

Option 2: Authorization Bearer Token

curl -H "Authorization: Bearer your_api_key_here" http://localhost:3333/symbols

Response Codes

  • 200 - Success
  • 401 - Unauthorized (missing or invalid API key)
  • 400 - Bad Request
  • 500 - Internal Server Error

πŸ“‘ API Documentation

Health Check (Public - No API Key Required)

GET /health

Symbol Management

# Add a symbol subscription
POST /symbols
{
  "exchange": "NASDAQ",
  "symbol": "AAPL", 
  "type": "quote"  # or "candle"
  "timeframe": "1D" # required for candle type
}

# Remove a symbol subscription  
DELETE /symbols
{
  "exchange": "NASDAQ",
  "symbol": "AAPL",
  "type": "quote"
}

# List active subscriptions
GET /symbols

# Get subscriptions by exchange/symbol
GET /symbols/NASDAQ/AAPL

Data Access

# Get real-time quote data
GET /quotes/NASDAQ/AAPL

# Get historical candlestick data
GET /candles/NASDAQ/AAPL/1D/100  # exchange/symbol/timeframe/limit

# Force reconnection
GET /reconnect

Technical Indicators

# Create a study session
POST /study-sessions
{
  "name": "My Analysis",
  "symbol": "NASDAQ:AAPL",
  "interval": "1D",
  "max_bars": 1000
}

# Add indicators to a session
POST /indicators
{
  "study_session_id": "session_uuid",
  "type": "RSI",
  "parameters": {"period": 14}
}

# Get indicator data
GET /indicators/{indicator_id}/data

πŸ— Architecture

Data Flow

  1. HTTP API β†’ Manages symbol subscriptions and data requests
  2. TradingView WebSocket Client β†’ Connects to TradingView real-time data
  3. Message Processing β†’ Routes and processes incoming market data
  4. Data Storage β†’ Quote data cached in Ristretto, candle data persisted to PostgreSQL
  5. API Response β†’ Serves cached quotes and historical candles via REST API

Database Schema

The service uses PostgreSQL with Ent ORM and includes these entities:

  • Symbol: Comprehensive symbol metadata with exchange, type, sector information
  • Candle: OHLCV data with quality indicators and market microstructure fields
  • ActiveSession: Real-time subscription tracking with status and error handling
  • StudySession: Technical analysis sessions for indicator calculations
  • Indicator: Individual technical indicators with parameters
  • IndicatorData: Time-series calculated indicator values

πŸ”§ Configuration

Configuration Methods (Priority Order)

  1. Command line flags: --server.port=3333
  2. Environment variables: PORT=3333
  3. Configuration file: config.yaml
  4. Default values

Complete Configuration Reference

server:
  port: "3333"
  host: "0.0.0.0"
  environment: "development"  # development, staging, production
  read_timeout: "30s"
  write_timeout: "30s"

database:
  host: "localhost"
  port: 5432
  user: "postgres"
  password: "secret"
  db_name: "tradingview"
  ssl_mode: "disable"
  max_open_conns: 25
  max_idle_conns: 5

tradingview:
  device_token: "required"
  session_id: "required"
  session_sign: "required"
  base_url: "https://www.tradingview.com"

websocket:
  url: "wss://prodata.tradingview.com/socket.io/websocket?from=screener%2F"
  max_retries: 5
  ping_interval: "30s"
  read_timeout: "60s"

cache:
  max_cost: 1073741824  # 1GB
  num_counters: 10000000
  default_ttl: "1h"

logging:
  level: "info"          # debug, info, warn, error
  format: "json"         # json, text

πŸ›  Development

Available Commands

make help           # Show all available commands
make dev            # Start development server with live reload
make build          # Build the application
make test           # Run tests
make test-cover     # Run tests with coverage
make lint           # Run linter
make fmt            # Format code
make clean          # Clean build artifacts
make docker-build   # Build Docker image

Project Structure

β”œβ”€β”€ cmd/
β”‚   β”œβ”€β”€ app/          # Main HTTP service application
β”‚   β”œβ”€β”€ example/      # Example usage
β”‚   └── test-config/  # Configuration testing tool
β”œβ”€β”€ internal/
β”‚   β”œβ”€β”€ config/       # Configuration management
β”‚   β”œβ”€β”€ dto/          # Data Transfer Objects
β”‚   β”œβ”€β”€ handler/      # HTTP API handlers
β”‚   └── service/      # Business logic services
β”œβ”€β”€ ent/              # Database schema and ORM (Ent)
β”œβ”€β”€ docs/             # Documentation
└── deploy/           # Deployment configurations

🐳 Docker Deployment

Using Docker Compose (Recommended)

The service includes a complete Docker Compose setup with PostgreSQL database:

# Navigate to docker directory
cd deploy/docker

# Copy environment template
cp .env.example .env

# Edit .env with your credentials
nano .env

# Start services (includes PostgreSQL + TradingView service)
docker-compose up -d

# View logs
docker-compose logs -f

# Stop services
docker-compose down

Using Docker Build

# Build Docker image
make docker-build

# Run with external database
docker run -p 3333:3333 \
  -e DB_HOST=your_db_host \
  -e DB_USER=postgres \
  -e DB_PASSWORD=your_password \
  -e DB_NAME=marketstream \
  -e TRADINGVIEW_DEVICE_TOKEN=your_token \
  -e TRADINGVIEW_SESSION_ID=your_session_id \
  -e TRADINGVIEW_SESSION_SIGN=your_signature \
  -e API_KEY=your_secure_api_key \
  -e AUTH_ENABLED=true \
  marketstream:latest

☁️ Coolify Deployment

Coolify is a self-hosted alternative to Heroku/Netlify. Deploy with these steps:

1. Fork/Clone Repository

git clone https://github.com/your-username/marketstream
cd marketstream

2. Create New Project in Coolify

  1. Login to your Coolify dashboard
  2. Create new project: "MarketStream"
  3. Choose "Docker Compose" as deployment type
  4. Set repository URL to your fork
  5. Set docker-compose file path: deploy/docker/docker-compose.yml

3. Configure Environment Variables

Add these secrets in Coolify dashboard:

TRADINGVIEW_DEVICE_TOKEN=your_device_token_here
TRADINGVIEW_SESSION_ID=your_session_id_here
TRADINGVIEW_SESSION_SIGN=your_session_signature_here
DB_PASSWORD=your_secure_postgres_password
API_KEY=your_secure_api_key_here

4. Deploy

  1. Click "Deploy" in Coolify dashboard
  2. Monitor deployment logs
  3. Service will be available at your configured domain
  4. Health check endpoint: https://your-domain.com/health

5. Post-Deployment

  • PostgreSQL data persisted in Docker volume
  • Automatic health checks configured
  • Auto-restart on failure enabled
  • Logs available in Coolify dashboard

πŸ§ͺ Testing

# Run all tests
make test

# Run tests with coverage
make test-cover

# Run benchmarks
make benchmark

# Test configuration
go run cmd/test-config/main.go

πŸ“Š API Examples

Subscribe to Apple Stock Quotes

curl -X POST http://localhost:3333/symbols \
  -H "Content-Type: application/json" \
  -H "X-API-Key: your_api_key_here" \
  -d '{"exchange":"NASDAQ","symbol":"AAPL","type":"quote"}'

Get Real-time Quote Data

curl -H "X-API-Key: your_api_key_here" http://localhost:3333/quotes/NASDAQ/AAPL

Subscribe to Bitcoin Candlestick Data

curl -X POST http://localhost:3333/symbols \
  -H "Content-Type: application/json" \
  -H "X-API-Key: your_api_key_here" \
  -d '{"exchange":"BINANCE","symbol":"BTCUSDT","type":"candle","timeframe":"1D"}'

Get Historical Candle Data

curl -H "X-API-Key: your_api_key_here" http://localhost:3333/candles/BINANCE/BTCUSDT/1D/50

πŸ“‹ Requirements

  • Go 1.24+
  • PostgreSQL 12+
  • TradingView credentials (device token, session ID, session signature)

πŸ”— Dependencies

🀝 Contributing

  1. Fork the repository
  2. Create a feature branch (git checkout -b feature/amazing-feature)
  3. Make your changes
  4. Run tests (make test)
  5. Commit your changes (git commit -m 'Add amazing feature')
  6. Push to the branch (git push origin feature/amazing-feature)
  7. Open a Pull Request

Development Guidelines

  • Follow Go best practices and idioms
  • Add tests for new functionality
  • Update documentation for API changes
  • Use the provided Makefile commands
  • Ensure all checks pass (make check)

πŸ“„ License

This project is licensed under the MIT License - see the LICENSE file for details.

πŸ”— Related Projects

πŸ†˜ Support

If you encounter any issues:

  1. Check the configuration examples
  2. Run configuration test: go run cmd/test-config/main.go
  3. Enable debug logging (LOG_LEVEL=debug)
  4. Check the TradingView WebSocket Client documentation
  5. Open an issue with detailed information

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published