Skip to content

vllm.config.structured_outputs

Classes:

StructuredOutputsConfig

Dataclass which contains structured outputs config for the engine.

Methods:

Attributes:

Source code in vllm/config/structured_outputs.py 
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
@config
class StructuredOutputsConfig:
    """Dataclass which contains structured outputs config for the engine."""

    backend: StructuredOutputsBackend = "auto"
    """Which engine will be used for structured outputs (e.g. JSON schema,
    regex, etc) by default. With "auto", we will make opinionated choices
    based on request contents and what the backend libraries currently support,
    so the behavior is subject to change in each release.
    The "lm-format-enforcer" backend is deprecated and will be removed in
    v0.35.0. Use "auto", "xgrammar", or "guidance" instead."""
    disable_any_whitespace: bool = False
    """If `True`, json output will always be compact without any whitespace.
    If `False`, the model may generate whitespace between JSON fields,
    which is still valid JSON. This is only supported for xgrammar
    and guidance backends."""
    disable_additional_properties: bool = False
    """If `True`, the `guidance` backend will not use `additionalProperties`
    in the JSON schema. This is only supported for the `guidance` backend and
    is used to better align its behaviour with `outlines` and `xgrammar`."""
    reasoning_parser: str = ""
    """Select the reasoning parser depending on the model that you're using.
    This is used to parse the reasoning content into OpenAI API format."""
    reasoning_parser_plugin: str = ""
    """Path to a dynamically reasoning parser plugin that can be dynamically
    loaded and registered."""
    enable_in_reasoning: bool = False
    """Whether to use structured input for reasoning."""

    def compile_factors(self) -> CompileFactors:
        """This config does not affect the compiled graph.

        If a field starts affecting it, return
        `get_compile_factors(self, ignored_factors)` with every other field in
        `ignored_factors`.
        """
        return {}

    @model_validator(mode="after")
    def _validate_structured_output_config(self) -> Self:
        if self.disable_any_whitespace and self.backend not in ("xgrammar", "guidance"):
            raise ValueError(
                "disable_any_whitespace is only supported for "
                "xgrammar and guidance backends."
            )
        if self.disable_additional_properties and self.backend != "guidance":
            raise ValueError(
                "disable_additional_properties is only supported "
                "for the guidance backend."
            )
        if self.backend == "lm-format-enforcer":
            logger.warning_once(
                "The lm-format-enforcer structured outputs backend is deprecated "
                "and will be removed in v0.35.0. Use auto, xgrammar, or guidance "
                "instead. See https://docs.vllm.ai/en/latest/features/"
                "structured_outputs/#migrating-from-lm-format-enforcer "
                "for migration guidance."
            )
        return self

backend = 'auto' class-attribute instance-attribute

Which engine will be used for structured outputs (e.g. JSON schema, regex, etc) by default. With "auto", we will make opinionated choices based on request contents and what the backend libraries currently support, so the behavior is subject to change in each release. The "lm-format-enforcer" backend is deprecated and will be removed in v0.35.0. Use "auto", "xgrammar", or "guidance" instead.

disable_additional_properties = False class-attribute instance-attribute

If True, the guidance backend will not use additionalProperties in the JSON schema. This is only supported for the guidance backend and is used to better align its behaviour with outlines and xgrammar.

disable_any_whitespace = False class-attribute instance-attribute

If True, json output will always be compact without any whitespace. If False, the model may generate whitespace between JSON fields, which is still valid JSON. This is only supported for xgrammar and guidance backends.

enable_in_reasoning = False class-attribute instance-attribute

Whether to use structured input for reasoning.

reasoning_parser = '' class-attribute instance-attribute

Select the reasoning parser depending on the model that you're using. This is used to parse the reasoning content into OpenAI API format.

reasoning_parser_plugin = '' class-attribute instance-attribute

Path to a dynamically reasoning parser plugin that can be dynamically loaded and registered.

compile_factors()

This config does not affect the compiled graph.

If a field starts affecting it, return get_compile_factors(self, ignored_factors) with every other field in ignored_factors.

Source code in vllm/config/structured_outputs.py 
47
48
49
50
51
52
53
54
def compile_factors(self) -> CompileFactors:
    """This config does not affect the compiled graph.

    If a field starts affecting it, return
    `get_compile_factors(self, ignored_factors)` with every other field in
    `ignored_factors`.
    """
    return {}