Skip to content

vllm.config.aux_output

Configuration for execution auxiliary outputs.

Classes:

AuxOutputConfig

Configuration for auxiliary-output delivery.

Methods:

  • compile_factors –

    WARNING: Whenever a new field is added to this config, add it to

Attributes:

Source code in vllm/config/aux_output.py 
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
@config
class AuxOutputConfig:
    """Configuration for auxiliary-output delivery."""

    enable_return_routed_experts: bool = False
    """Capture and return routed-experts auxiliary outputs."""

    max_bytes: int | None = Field(default=None, gt=0)
    """Initial capacity, or ``None`` to derive it from the KV cache capacity.
    Grows when every object is referenced."""

    @property
    def enabled(self) -> bool:
        """Whether any execution auxiliary output is enabled."""
        return self.enable_return_routed_experts

    def compile_factors(self) -> CompileFactors:
        """WARNING: Whenever a new field is added to this config, add it to
        `ignored_factors` if it does not affect the computation graph.
        """
        ignored_factors = {
            "max_bytes",
        }
        return get_compile_factors(self, ignored_factors)

enable_return_routed_experts = False class-attribute instance-attribute

Capture and return routed-experts auxiliary outputs.

enabled property

Whether any execution auxiliary output is enabled.

max_bytes = Field(default=None, gt=0) class-attribute instance-attribute

Initial capacity, or None to derive it from the KV cache capacity. Grows when every object is referenced.

compile_factors()

WARNING: Whenever a new field is added to this config, add it to ignored_factors if it does not affect the computation graph.

Source code in vllm/config/aux_output.py 
26
27
28
29
30
31
32
33
def compile_factors(self) -> CompileFactors:
    """WARNING: Whenever a new field is added to this config, add it to
    `ignored_factors` if it does not affect the computation graph.
    """
    ignored_factors = {
        "max_bytes",
    }
    return get_compile_factors(self, ignored_factors)