AMD has made its first foray into the world of small language models with the launch of AMD-135M, showcasing the company's growing ambitions in the AI space. This new model aims to provide efficient AI capabilities for businesses while leveraging AMD's hardware strengths.
Key highlights of AMD-135M:
- Two variants : AMD-Llama-135M for general use and AMD-Llama-135M-code optimized for coding tasks
- Training process :
- Base model trained on 670 billion tokens over 6 days
- Code variant fine-tuned with additional 20 billion tokens over 4 days
- Used four 8-way AMD Instinct MI250-based nodes for training
- Speculative decoding : Employs a smaller draft model to generate multiple candidate tokens simultaneously, verified by a larger target model
- Performance claims : AMD reports significant speedups on its hardware compared to inference without speculative decoding
The introduction of AMD-135M signals the company's intent to compete in the AI model space, potentially challenging NVIDIA's dominance. By focusing on small language models, AMD is targeting a niche that may be particularly valuable for businesses requiring on-premises AI solutions with lower computational demands.
AMD's approach of open-sourcing the training code, dataset, and weights for AMD-135M could foster collaboration and innovation in the AI community. This move aligns with the growing trend of more accessible and transparent AI development.
While the performance claims are promising, it's worth noting that the benchmarks were conducted by AMD itself. Independent testing will be crucial to validate these results across different scenarios and hardware configurations.
As AMD continues to develop its AI portfolio, including both hardware and software offerings, the tech industry will be watching closely to see how this impacts the competitive landscape and drives innovation in AI technologies.