Native structured outputs with a dynamic schema

[cberes]

I'm currently using OpenAI's API with a json_schema response format. My understanding is that Spring AI's native structured outputs is the equivalent, so when I switch to Spring AI I'd like to use native structured outputs. However I make the schema dynamically. Is that something I can do with Spring AI?

The schema looks something like this, but as I said it's dynamic. The fields in AiResponse may be different each time. ValueType is one of the 4 subtypes below, except we don't actually have a MyEnum; we build the schema's enum property dynamically.

class AiResponse {
  // the actual fields here are different each time
  FieldValue fieldId1;
  FieldValue fieldId2;
}

class FieldValue {
  ValueType value;
  String reason;
  int confidence;
}

class NumberValueType implements ValueType {
  int value;
}

class StringValueType implements ValueType {
  String value;
}

class EnumValueType implements ValueType {
  MyEnum value;
}

class MultiEnumValueType implements ValueType {
  Set<MyEnum> value;
}

I found a way to do this where I build the schema by hand using Jackson. And also I use Jackson's readTree to get a JsonNode, then extract the fields manually. Below is an example.

My understanding is that I have to use a BeanOutputConverter if I want to use native structured outputs. Is that correct?

Is there a better or simpler way to do this? I worry that this won't be portable to other LLMs if I have to build the schema by hand.

public <T> T doStructuredOutputsThings(String prompt, JsonNode schema, Function<JsonNode, T> converter, Class<T> type) {
  var chatClient = chatClientService.chatClient();
  var response = chatClient.prompt()
      .advisors(AdvisorParams.ENABLE_NATIVE_STRUCTURED_OUTPUT)
      .user(prompt)
      .call()
      .responseEntity(new BeanOutputConverter<>(type) {
        @Override
        public T convert(String text) {
          try {
            return converter.apply(getObjectMapper().readTree(text));
          } catch (JsonProcessingException e) {
            throw new RuntimeException(e);
          }
        }

        @Override
        public String getJsonSchema() {
          try {
            return getObjectMapper().writeValueAsString(schema);
          } catch (JsonProcessingException e) {
            throw new RuntimeException(e);
          }
        }
      });

  return response.entity();
}

[weilhuang]

@cberes If you only target one model/provider that supports native structured output, setting outputSchema(...) directly may be simpler.If you keep the custom BeanOutputConverter, I would also override getFormat(), not only getJsonSchema(), so the fallback prompt-based structured output path stays consistent as well.

[adrianodpdiaz]

Yes, you can pass a dynamically generated JSON Schema directly to Spring AI and you don't need to subclass BeanOutputConverter just to make the native structured-output schema dynamic.

Spring AI exposes the schema directly through StructuredOutputChatOptions, so if you already have the JSON Schema as a JsonNode/String, you can pass it directly:

String jsonSchema = objectMapper.writeValueAsString(schema);

var options = StructuredOutputChatOptions.builder()
        .outputSchema(jsonSchema)
        .build();

var response = chatClient.prompt()
        .options(options)
        .user(prompt)
        .call()
        .content();

JsonNode result = objectMapper.readTree(response);

Provider-specific options can also expose the same outputSchema(...) functionality, e.g. OpenAiChatOptions.

The BeanOutputConverter is useful when the output maps to a known Java type, because it generates the JSON Schema from that type and converts the response back to the Java object. For a response whose fields and enum values are genuinely dynamic, building the schema yourself and parsing the result as a JsonNode is a better fit.

For portability, StructuredOutputChatOptions gives you the common Spring AI abstraction, but the schema itself is still subject to the capabilities and restrictions of the target provider/model. Spring AI's native structured-output support does not make every JSON Schema feature portable across providers. For example, the supported JSON Schema subset can differ between providers.