Open source Machine Learning Infrastructure tools
22 free, open source options sorted by GitHub stars.
OllamaRun open-source LLMs locally on your own machinellama.cppRun LLMs locally with minimal setup, maximum hardware supportvLLMHigh-throughput, memory-efficient serving engine for LLMsGPT4AllRun open-source AI models privately on your own deviceLocalAISelf-hosted AI runtime for text, voice, vision, and agentsJanRun AI models locally or connect to cloud, privatelyOpen-NotebookAI-powered research notebook with full privacy controlLangfuseOpen source LLM engineering platform for AI-powered applicationsArize PhoenixOpen-source LLM tracing & evaluation for AI optimizationOpenLLMetryMonitor LLM performance with open-source observabilityHeliconeBuild reliable AI apps with comprehensive observabilityLangWatchSimulation-based testing and evaluation for AI agentsAgentaBuild, automate, and improve AI agents with your teamLatitudeFull observability for AI agents in productionLaminarAI-powered platform for engineering LLM productsDeepnotePython, SQL, and AI in a collaborative data notebookOpenLIT Monitor, debug, and scale LLM applications with easedstackGPU orchestration across clouds, Kubernetes, and on-premBeamServerless GPU compute with sub-second cold startsmlopExperiment tracking platform for ML teamsSpanlensLLM observability: cost, latency, and traces in one placeGreptureTrace, evaluate, and protect every LLM call in one SDK