Forge CLI

Forge CLI is a swarm agent system and automated AI optimization engine designed to maximize GPU inference performance for machine learning models. This tool is built for ML teams, infrastructure engineers, and enterprises who need to accelerate AI model inference at scale without manual low-level optimization work. Forge automatically generates optimized GPU kernels from any PyTorch or HuggingFace model, achieving up to 5x faster inference than torch.compile with 97.6% correctness. Its core value lies in transforming underutilized GPU resources into highly efficient compute engines, directly addressing the critical compute problem where GPUs often operate at only 16% utilization, leading to significant wasted expenditure and power.

The concrete problem Forge solves is the severe underutilization of expensive GPU hardware, where only about 16% of GPU cycles perform real compute work while the rest are lost to idle cycles, memory stalls, and unoptimized operations. This inefficiency matters profoundly to users because it translates to paying for 100% of GPU capacity while receiving minimal productive output, wasting approximately $840K per $1M spent on GPU infrastructure. For enterprises running large-scale AI inference, such as with models like Qwen3-235B, this inefficiency directly impacts latency, throughput, and operational costs, preventing real-time responses and scalable user serving. Forge directly tackles this by optimizing kernels to boost utilization to around 88%, turning wasted cycles into valuable computation and delivering substantial cost savings and performance gains.

One major feature group is Forge's automated kernel optimization for any AI model architecture, including language models, image generation, and speech recognition. The system works by analyzing the model's operations and generating CUDA or Triton optimized kernels specifically tailored for the user's GPU setup, such as NVIDIA datacenter GPUs like B200, H200, H100, L40S, and A100. This is useful because it eliminates the need for manual low-level coding, automatically transforming unoptimized kernels that exhibit poor SM utilization and memory I/O patterns into highly efficient ones. The result is a drop-in replacement requiring zero code changes, providing the same API but with dramatically improved inference speed and GPU utilization, as demonstrated by cutting Qwen3's 235B MoE attention layer token generation latency by 7.6x.

A second major feature group is the performance and correctness guarantees, including manual verification for 100% numerical correctness against the original model. Forge delivers optimized kernels in under an hour, with every output undergoing rigorous verification to ensure it produces identical results to the unoptimized version. This feature is critical for production deployments where model accuracy cannot be compromised, guaranteeing that the speed improvements—such as increasing throughput from 312 to 3,180 tokens per second—do not come at the expense of reliability. The system also provides concrete metrics like 90% cost reduction per million tokens by maximizing GPU utilization from 24% to 95%, offering users predictable and verified performance uplifts on their target hardware.