#large language models

Building Aligned Intelligence System. Part I: Creating GPT Assistant

In recent years, the field of natural language processing has witnessed a remarkable breakthrough with the advent of Large Language Models (LLMs). These models have demonstrated unprecedented performance in a wide range of language tasks, from text generation to question answering. However, the mathematical formulation of these models and the techniques used to fine-tune them for specific ta...

AstraBlog · EN

Building Aligned Intelligence System. Part II. Improving Large Language Models

In this post we will look at different techniques for steering LLMs behaviour to get desired outcomes, starting with some basic general principles such as writing a good prompt and ending with fine-tuning and augmenting models with external knowledge. Methods discussed in this post are mainly aimed at improving LLMs reliability and ensuring the consistency and factual accuracy of their...

AstraBlog · EN

Exploring Parallel Strategies with Jax

Training large language models either like GPT, LlaMa or Mixtral requires immense computational resources. With model sizes ballooning into the billions or sometimes even trillions of parameters, specialized parallelization techniques are essential to make training feasible. In this post, we’ll explore implementing some of these scaling strategies in Jax - a Python framework designed for hig...

AstraBlog · EN

Transformers Inference Optimization Toolset

Large Language Models are pushing the boundaries of artificial intelligence, but their immense size poses significant computational challenges. As these models grow, so does the need for smart optimization techniques to keep them running efficiently on modern hardware. In this post, we’ll explore key optimization strategies that are making LLMs faster and more memory-efficient. We’ll star...

AstraBlog · EN

Speculative Decoding: The Art of Guessing Well

Large Language Models spend most of their decoding time not computing, but waiting: every new token requires reading all of the model weights from memory, only to multiply them by a single vector. What if the model could check several tokens for the price of one? In this post we’ll derive speculative decoding from first principles, build a general framework that covers every modern drafte...

AstraBlog · EN

Article: Systemic Approach to Memory, Knowledge, and Context in Agentic AI Architectures

The article argues that reliable agentic AI architecture depends on a context engine that selects, injects, compresses, and expires information before it reaches the model. It separates context, memory, and knowledge, and recommends tiered memory, curated knowledge, and token budgeting. It also covers common anti-patterns, such as context stuffing and stale data loops, with mitigations. By Adi Polak

InfoQ · EN

Google and Meta Open-source TorchTPU to Bring Native PyTorch Support to Google TPUs

TorchTPU is an open-source project developed by Google in collaboration with Meta that provides a native PyTorch backend for Google TPUs. It enables PyTorch workloads to execute on TPU hardware with minimal code modifications, supporting eager and compiled execution, distributed computing, and hardware-specific optimizations. By Sergio De Simone

InfoQ · EN