Note: That the below is a continuous work in progress. This blog’s ethos of writing posts is elaborated here. All changes can be tracked via the commit history of the markdown file here.

Distributed Agents and what it means for foundational models?

The next step of the AI paradigm is a push towards distributed, federated AI agent networks. This is a result of a current bottleneck in AI adoption: the best models are controlled by a handful of companies, and these models have poor data control limits (that is, there is a chance sensitive data is absorbed into the training data of those foundational models). An obvious workaround would be to create your own model, either by training one from scratch or fine-tuning an open-source model locally (using Ollama, for example). A problem with this solution is that it requires an immense amount of resources to build, host, and maintain these models. Many companies and developers are not positioned to take on such a proposition.

In order to compromise between performance and meaningful data control, there is a requirement for entities (companies or individuals) to host their own local models. However, to achieve a sufficient threshold of performance (especially in complex reasoning tasks), these entities must retain the ability to communicate with larger frontier foundation models. This problem is fundamentally a distributed systems problem: there is a need for communication protocol(s) capable of addressing classic and emergent distributed systems challenges: dynamic resource allocation needed to maximize the efficiency of a heterogeneous cluster of local LLMs, hierarchical delegation structures, fault tolerance against Byzantine failures or node preemption, and resilience to network partitions between local LLMs. Initial steps in this direction include Google’s A2A protocol and the Minions protocol.

In the following sections, I will break down both protocols, examine related research explorations, deep-dive into future protocol architectures, provide full citations, and summarize comparisons in an appendix.

Table of Contents

• Minions Protocol
• A2A Protocol
• Related Explorations
• Future Protocols
• Citations
• Appendix

Minions Protocol

The Minions protocol explores reducing inference costs on data-intensive reasoning tasks through collaborative execution between on-device (local/on-prem) language models and cloud frontier models.

In the paper, the authors demonstrate up to a 30.4x reduction in inference cost while sacrificing only 9.4 accuracy points on average compared to querying a single frontier model throughout. The core contribution is the Minions communication protocol, designed to tackle reasoning tasks where the context greatly exceeds the context window (or budget) of an individual model. Remarkably, Minions answers queries without requiring the remote frontier model to ever ingest the full raw context. Beyond avoiding wasted bandwidth and memory, this architecture provides structural protection against data leakage. While privacy preservation was not the primary evaluation metric of the original Minions paper, related investigations in collaborative inference demonstrate similar shielding properties [3], [4]. A natural next frontier is extending the protocol beyond long-form text documents to high-density multimodal streams (audio, image, and video).

The authors first analyze a naive baseline called Minion, which relies solely on unstructured natural language chatter between local and remote models. This baseline degrades quickly: smaller local models struggle to track long multi-step instructions and become erratic when context sizes escalate past 65k tokens.

To overcome this, the authors introduce the full Minions protocol, structured around an iterative three-phase loop:

• Job preparation on remote
• Job execution and filtering locally
• Job aggregation on remote

In many respects, this workflow directly mirrors the classic MapReduce paradigm in distributed computing.

Job Preparation on Remote

The frontier model generates executable code that yields a list of job specifications for local models to evaluate concurrently. Each job specification, \mathbf{t}^{i}, is a query-context pair:

\mathbf{t}^{i} = (q^i, c^i)

where q^i is the task query for the i-th sub-task and c^i is the partial context window slice required to resolve it. How can the frontier LLM determine the required context slice without holding the full document in memory?

It does so through a two-stage planning routine: the frontier model first decomposes the user’s high-level task into a general plan, and then synthesizes a Python chunking and routing function. When this function runs locally, it segments the context and emits discrete, self-contained sub-jobs. Mapping these local models to their assigned sub-jobs functions as the Map phase of the pipeline.

Job Execution and Filtering Locally

These sub-jobs are executed in parallel across local LLMs. For each job, the local model decides whether to abstain (if the chunk is irrelevant) or emit a structured JSON object, z^{(i)}, containing fields for explanation, citation, and answer:

z^{(i)} = \{\text{explanation}, \text{citation}, \text{answer}\}

Abstaining filters noise early, saving upstream memory and communication overhead, while the citation field grounds the model’s intermediate reasoning. These emitted objects are then joined into an aggregated context payload w.

Job Aggregation on Remote

The remote frontier model receives the aggregated payload w alongside a synthesis prompt p_{\text{synthesize}}. It outputs a structured JSON response containing a decision directive (CONTINUE or FINISH) and the response candidate. If further reasoning or clarification is required, the frontier model issues a new round of job specifications; otherwise, it emits the final answer to the user. This aggregation and synthesis phase corresponds directly to the Reduce phase of MapReduce.

Scaling Parallel Workloads On-Device

Empirically, scaling the on-device workload correlates positively with overall system accuracy: increasing the task count per round, sampling multiple candidate trajectories per local model, and adopting finer chunk granularities all elevate reasoning quality. However, this incurs additional on-device latency and VRAM footprint. Balancing this trade-off is critical to prevent saturating local compute or overwhelming the frontier model’s synthesis stage with marginal responses.

A2A Protocol

The Model Context Protocol (MCP) has rapidly become the standard for exposing tools, APIs, and databases to LLMs. However, within MCP’s design, every external entity is modeled as a tool. When applied to multi-agent architectures, this tool-centric abstraction breaks down: agents require reciprocal, peer-to-peer relationships, stateful handoffs, and bidirectional streaming rather than one-directional RPC tool calls.

Google’s Agent-to-Agent (A2A) protocol addresses this gap by defining a dedicated communication substrate for inter-agent collaboration. Agents run behind dedicated endpoints that expose their identity, supported modalities, and operational interfaces. The protocol supports multiple interaction patterns tailored for distributed workloads:

• Polling: Lightweight state inspection for synchronous short-lived tasks.
• Server-Sent Events (SSE): Unidirectional real-time event streaming for continuous token or trace outputs.
• Push Notifications (Webhooks): Asynchronous completion signals for long-running batch or offline workflows.

Communication in A2A utilizes JSON-RPC 2.0 payloads:

class JSONRPCMessage(BaseModel):
    jsonrpc: Literal['2.0'] = Field('2.0', title='Jsonrpc')
    id: Optional[Union[int, str]] = Field(None, title='Id')


class JSONRPCRequest(BaseModel):
    jsonrpc: Literal['2.0'] = Field('2.0', title='Jsonrpc')
    id: Optional[Union[int, str]] = Field(None, title='Id')
    method: str = Field(..., title='Method')
    params: Optional[Dict[str, Any]] = Field(None, title='Params')


class JSONRPCResponse(BaseModel):
    jsonrpc: Literal['2.0'] = Field('2.0', title='Jsonrpc')
    id: Optional[Union[int, str]] = Field(None, title='Id')
    result: Optional[Dict[str, Any]] = Field(None, title='Result')
    error: Optional[JSONRPCError] = None


class PushNotificationConfig(BaseModel):
    url: str = Field(..., title='Url')
    token: Optional[str] = Field(None, title='Token')
    authentication: Optional[AuthenticationInfo] = None


class TaskPushNotificationConfig(BaseModel):
    id: str = Field(..., title='Id')
    pushNotificationConfig: PushNotificationConfig

In raw A2A, validating payload schemas across complex choreography can be tricky. Community toolkits such as legion-a2a provide Pydantic validators over these interaction patterns. When scaling to frameworks like CrewAI or LangGraph, developers often bridge A2A interfaces directly into internal event buses.

The Agent Card: Dynamic Capability Discovery

The core innovation of A2A is the Agent Card, a standardized metadata contract that declares an agent’s operational capabilities, supported input/output modalities, authentication schemas, and discrete skills.

# Agent Card Definition
class AgentCard(BaseModel):
    name: str
    description: str | None = None
    url: str
    provider: AgentProvider | None = None
    version: str
    documentationUrl: str | None = None
    capabilities: AgentCapabilities
    authentication: AgentAuthentication | None = None
    defaultInputModes: list[str] = ['text']
    defaultOutputModes: list[str] = ['text']
    skills: list[AgentSkill]

# Agent Card Instantiation Example
capabilities = AgentCapabilities(streaming=True)
skills = [
    AgentSkill(
        id='download_closed_captions',
        name='Download YouTube Closed Captions',
        description='Retrieve closed captions/transcripts from YouTube videos',
        tags=['youtube', 'captions', 'transcription', 'video'],
        examples=[
            'Extract the transcript from this YouTube video: https://www.youtube.com/watch?v=dQw4w9WgXcQ',
            'Download the captions for this YouTube tutorial',
        ],
    )
]

agent_card = AgentCard(
    name='YouTube Captions Agent',
    description='AI agent that extracts closed captions and transcripts from YouTube videos for downstream processing.',
    url=f'http://{host}:{port}/',
    version='1.0.0',
    defaultInputModes=['text'],
    defaultOutputModes=['text'],
    capabilities=capabilities,
    skills=skills,
)

The Agent Card tackles a fundamental problem that traditional distributed systems never had to solve. Classic distributed architectures assume hardware homogeneity or well-defined deterministic benchmarks: a node is quantified by CPU cores, RAM, and network bandwidth, metrics that remain static and predictable. Nodes never need to negotiate semantic competence.

In Multi-Agent Systems (MAS), two worker nodes running identical quantized 7B models on identical GPUs can produce vastly different quality outputs depending on their system prompts, fine-tuning histories, context-window compression, or retrieval sources. For example, two transcription agents might boast identical throughput, but one systematically drops domain jargon while the other excels at technical acronyms. Current protocols lack expressive ways to describe, measure, and calibrate this semantic variance.

For practical walkthroughs implementing A2A servers, see Google’s tutorials on Integrating Ollama into an A2A Server and Authoring Agent Skills.

Related Explorations

The convergence of distributed computing, edge inference, and multi-agent coordination has sparked diverse research efforts. The following projects and papers provide key building blocks:

1. Long-Context Decomposition & MapReduce Workflows

• LLM × MapReduce (Lee et al., 2024) [8]: Formalizes document-scale information extraction by embedding LLM inference directly into Map and Reduce operators. Demonstrates that structured extraction can be executed efficiently on constrained local infrastructure through algorithmic chunking and aggregation.
• Chain of Agents (CoA) (Zhang et al., NeurIPS 2024) [9]: Deploys an ensemble of worker agents over segmented text chunks, passing a shared communication vector along a sequential chain. CoA mitigates lost-in-the-middle context degradation and scales horizontally across multiple smaller workers.
• Self-Taught Agentic Long Context Understanding (2025) [11]: Explores how autonomous agents can iteratively train their own chunking, retrieval, and navigation strategies across massive contexts without hand-crafted heuristics.

2. Multi-Agent Coordination & Heterogeneous Protocols

• When One LLM Drools, Multi-LLM Collaboration Rules (Feng et al., 2025) [7]: A position paper outlining a comprehensive taxonomy for multi-LLM networks. It categorizes model interactions across four distinct communication abstractions: API-level, text-level, logit-level, and weight-level, detailing when tight coupling (logits/weights) outperforms loose text-based messaging.
• Role-RL (2024) [10]: Formulates role assignment across heterogeneous LLMs using online reinforcement learning, matching distinct sub-tasks to the model best calibrated to solve them while minimizing computational cost.

3. Agent Execution Engines, Memory & Tool Grounding

• smolagents (Hugging Face) [15]: An ultra-lightweight agent framework championing Code Agents, where LLMs communicate and invoke actions by writing clean, executable Python snippets rather than verbose JSON schemas. Code actions drastically reduce token bloat and enable efficient local sandboxed execution.
• R2D2: Reflective Agentic Memory (Huang et al., 2025) [12]: Integrates replay-buffer memory with reflective evaluation mechanisms, allowing agents to replay past navigational trajectories and adjust their strategies dynamically.
• Attention-Driven GUI Grounding (TAG) (Xu et al., AAAI-25) [13]: Demonstrates tuning-free grounding for multimodal models by aggregating self-attention maps, allowing edge vision-language models to interact accurately with user interfaces without resource-heavy fine-tuning.
• Laminar / Index: High-performance vector indexing and retrieval infrastructure purpose-built for agent memory and real-time execution pipelines.
• Chefs’ Random Tables (Likhosherstov et al., NeurIPS 2022) [14]: Explores efficient non-trigonometric random feature approximations, foundational for low-latency kernel methods and distributed representations on resource-constrained devices.

Future Protocols

Combining on-device processing with frontier cloud intelligence exposes uncharted territory at the intersection of distributed systems, cryptography, and multi-agent reinforcement learning. Below are four primary architectural frontiers for next-generation edge-cloud agent protocols.

1. Multi-Agent Co-Training & Edge Quorums (MARL)

Current multi-agent frameworks predominantly orchestrate pre-trained, frozen models out of the box using prompt templates. However, prompting frozen models creates brittle interactions: models frequently misinterpret peers, hallucinate intent, or fail to converge during multi-turn negotiations.

A promising path forward is adapting cooperative multi-agent reinforcement learning frameworks such as MAPoRL (Multi-Agent Post-Co-Training for Collaborative LLMs) [6]. Instead of training models in isolation, a cluster of edge Small Language Models (SLMs) can be post-trained collaboratively:

• Local Quorum Deliberation: Before escalating a task to an expensive cloud model, local edge models engage in a multi-turn, verified discussion protocol. Models propose candidate hypotheses, critique counter-arguments, and vote to form a local consensus.
• Verifier-Guided Communicative Rewards: By rewarding concise, corrective communication during training, the edge ensemble learns to maximize consensus quality while minimizing the internal token budget.
• Egress Token Minimization: When edge quorums successfully resolve or filter candidate sub-queries locally, they eliminate unnecessary cloud API round trips. If escalation is required, they emit a concise, verified digest rather than uncurated raw tokens, drastically slashing bandwidth and inference costs.

2. Multi-Layer Inter-Agent Communication: Beyond JSON-RPC

Natural language and JSON-RPC text messaging are convenient abstractions, but they are computationally inefficient communication channels. Transmitting serialized ASCII characters across nodes forces continuous tokenization and detokenization overhead.

Building on the collaboration taxonomy established by Feng et al. [7], future protocols can operate across tighter coupling layers:

• Logit-Level & Latent Vector Exchange: When local models share architectural families or tokenizers, exchanging top-k logits or intermediate hidden embeddings conveys probability mass and confidence calibrations far more densely than serialized text strings.
• Distributed Speculative Decoding: Local SLMs can serve as speculative drafting engines running continuously at low power on consumer NPUs/GPUs. They speculate several candidate reasoning steps ahead; the remote frontier model validates or corrects the entire token tree in a single cloud forward pass, slashing latency without sacrificing reasoning rigor.
• Privacy-Preserving Delegation: As shown in PAPILLON [3] and CoGenesis [4], local models can execute privacy transforms, scrub Personally Identifiable Information (PII), or synthesize semantic sketches before payloads depart local boundaries. Formalizing differential privacy guarantees into agent communication protocols ensures verifiable protection against model memorization and prompt extraction attacks [5].

3. Classical Distributed Systems Guarantees in Agent Topologies

Merging the Minions execution loop with A2A services creates distributed topologies that must confront classical systems challenges:

• Fault Tolerance and Fail-Stop Recovery: In edge networks, local nodes drop offline unexpectedly (e.g., mobile devices switching networks, laptop lids closing, preemptible cloud instances). If a worker fails during a local Map phase, the orchestrator cannot stall indefinitely. Protocols require heartbeat mechanisms, leased job assignments with exponential-backoff retries, and idempotent job execution to guarantee forward progress under high node churn.
• Byzantine Faults & Hallucination Mitigation: In an untrusted or heterogeneous network, an agent might experience catastrophic hallucination, corrupted context, or adversarial prompt injection. Protocols must implement Byzantine fault-tolerant consensus: verifying worker claims through cross-agent validation, zero-knowledge proofs of execution (zk-VMs), or consistency thresholds before incorporating outputs into global memory.
• Hierarchical Topologies (Edge → Fog → Cloud): Rather than flat two-tier star topologies (where one cloud model manages N edge devices), production systems will naturally evolve into multi-tier trees and Directed Acyclic Graphs (DAGs). Micro-models on sensor/mobile nodes stream to local on-prem gateway servers (Fog tier), which synthesize domain-specific clusters before forwarding aggregated payloads to frontier cloud models.

4. Dynamic Resource Allocation as Contextual Bandits

On-device inference is fundamentally resource-constrained. Local devices experience thermal throttling, fluctuating VRAM headroom, battery drain, and network latency jitter. Static assignment of sub-jobs to local models is suboptimal.

Instead, sub-task dispatching can be formulated as a Contextual Multi-Armed Bandit (MAB) or a continuous combinatorial auction:

• State Context: At scheduling time, each candidate edge node reports its current system state (available VRAM, compute utilization, battery state, thermal limits) along with its domain competence profile recorded in its extended Agent Card.
• Utility Function: The scheduling policy balances a multi-objective utility function:

U = \alpha \cdot \text{Accuracy} - \beta \cdot \text{Cost} - \gamma \cdot \text{Latency} - \delta \cdot \text{PrivacyRisk}

• Adaptive Exploration: By tracking historical task completion rates, confidence scores, and error patterns, the orchestrator dynamically routes sub-jobs to the most cost-effective and reliable local nodes, reserving frontier cloud queries strictly for high-uncertainty tasks.

Citations

[1] Bansal, H., et al. (2025). Minions: Fast, Cheap, and Decentralized Reasoning via Edge-Cloud LLM Collaboration. arXiv preprint arXiv:2502.15964.

[2] Google. (2025). Agent-to-Agent (A2A) Protocol Specification and Tutorials. Google A2A Documentation.

[3] Li, S., Raghuram, V. C., Khattab, O., Hirschberg, J., & Yu, Z. (2024). PAPILLON: Privacy Preservation from Internet-based and Local Language Model Ensembles. arXiv preprint arXiv:2410.17127.

[4] Zhang, K., Wang, J., Hua, E., Qi, B., Ding, N., & Zhou, B. (2024). CoGenesis: A Framework Collaborating Large and Small Language Models for Secure Context-Aware Instruction Following. Findings of the Association for Computational Linguistics (ACL 2024). arXiv:2403.03129.

[5] Carlini, N., et al. (2024). Uncovering Latent Memories: Assessing Data Leakage and Memorization Patterns in Frontier AI Models. arXiv preprint arXiv:2406.14549.

[6] Zhang, Y., et al. (2025). MAPoRL: Multi-Agent Post-Co-Training for Collaborative Large Language Models with Reinforcement Learning. arXiv preprint arXiv:2502.18439.

[7] Feng, S., Ding, W., Liu, A., Wang, Z., Shi, W., Wang, Y., Shen, Z., Han, X., Lang, H., Lee, C.-Y., Pfister, T., Choi, Y., & Tsvetkov, Y. (2025). When One LLM Drools, Multi-LLM Collaboration Rules. arXiv preprint arXiv:2502.04506.

[8] Lee, J., et al. (2024). LLM × MapReduce: Simplified Long-Sequence Processing using Large Language Models. arXiv preprint arXiv:2410.09342.

[9] Zhang, Y., Sun, R., Chen, Y., Pfister, T., Zhang, R., & Arik, S. Ö. (2024). Chain of Agents: Large Language Models Collaborating on Long-Context Tasks. Advances in Neural Information Processing Systems (NeurIPS 2024), 37. NeurIPS Proceedings.

[10] Wang, H., et al. (2024). Role-RL: Online Long-Context Processing with Role Reinforcement Learning for Distinct LLMs in Their Optimal Roles. arXiv preprint arXiv:2409.18014.

[11] Liu, X., et al. (2025). Self-Taught Agentic Long Context Understanding. arXiv preprint arXiv:2502.15920.

[12] Huang, T., Basu, K., Abdelaziz, I., Kapanipathi, P., May, J., & Chen, M. (2025). R2D2: Remembering, Replaying and Dynamic Decision Making with a Reflective Agentic Memory. arXiv preprint arXiv:2501.12485.

[13] Xu, H., Chen, Q., Wang, L., & Liu, L. (2025). Attention-Driven GUI Grounding: Leveraging Pretrained Multimodal Large Language Models Without Fine-Tuning. Proceedings of the AAAI Conference on Artificial Intelligence (AAAI-25), 39(8), 8851–8859. AAAI Proceedings.

[14] Likhosherstov, V., Choromanski, K. M., Dubey, K. A., Liu, F., Sarlos, T., & Weller, A. (2022). Chefs’ Random Tables: Non-Trigonometric Random Features. Advances in Neural Information Processing Systems (NeurIPS 2022), 35. NeurIPS Proceedings.

[15] Hugging Face. (2025). smolagents: A minimalist library for building code-based autonomous agents. GitHub Repository.

Appendix

Protocol & Architectural Comparison