OpenSearch Flattened Field Type in Java: A Practical Guide
Are you teetering on the edge of a mapping explosion or suffering with performance issues from too many field mappings? Be it strict or dynamic mappings, Opensearch has its limits. Don’t worry though, the OpenSearch flattened field type (OS flat object, ES flattened) is OpenSearch’s solution to this challenge and we’ll explore how to use it effectively with Java in today’s blog post.
This blog post demonstrates the flattened field type (OS flat object, ES flattened) using code examples from the Code Sloth Code Samples repository. You can find the complete samples in GitHub via our Code Sloth Code Samples page.
Note: other than some small syntax differences, the OpenSearch and Elasticsearch implementations are very similar. This article will refer to OpenSearch but link to both implementations, so feel free to adapt to Elasticsearch if that is your Engine of choice!
Let’s dive in! 🦥
What’s to Come
In this post, we’ll explore:
- What is the Flattened Field Type? – Understanding the basics of how flattened fields work
- Why Use Flattened Fields? (Mitigating Mapping Explosion) – The problems flattened fields solve
- The Basics: Simple Flattened Field Mapping – Creating your first flattened field mapping
- Observing Keywords: How Flattened Fields Index Sub-Properties – Verifying that sub-properties are indexed as single keyword tokens
- Searching on Flattened Fields – How to search on flattened field properties using dotted notation
- Matching Multiple Properties from the Same Flattened Field – Combining search conditions
- Multiple Flattened Fields in a Single Document – Working with multiple flattened fields
- Nested Flattened Fields: Multi-Level Dotted Notation – Handling deeply nested structures
- Flattened Arrays: Understanding the Limitations – Important constraints when working with arrays
- Flattened vs. Nested: When to Use Each – Choosing the right approach for your use case
- Sloth Summary – Key takeaways and recommendations
What is the Flattened Field Type?
The flattened field type (OS flat object, ES flattened) in OpenSearch is a specialized mapping type that treats an entire object as a single field. Instead of creating a separate field mapping for each property within an object, flattened fields index all sub-properties as keywords under a single field name. Pretty neat, right? This allows you to search on object properties using dotted notation (like metadata.title or metadata.brand) without the mapping overhead of traditional object types.
From the OpenSearch documentation, flattened fields are particularly useful when:
- You have objects with unknown or frequently changing properties
- You want to avoid mapping explosion (where dynamic mappings create too many field mappings)
- You need to search on object properties without full object type functionality
Why Use Flattened Fields? (Mitigating Mapping Explosion)
The biggest reason to use flattened fields? They prevent mapping explosion. Here’s the thing: when you use the default object type in OpenSearch with dynamic mapping enabled (which is the default behavior), each unique property within an object automatically gets its own mapping entry. This happens because dynamic mapping creates new field mappings on-the-fly as documents are indexed with previously unseen properties.
If you’re indexing documents with dynamic or highly variable object structures—or even with dynamic templates that automatically map certain field patterns—this can quickly lead to:
- Mapping explosion: Thousands of field mappings created for unique property combinations
- Memory overhead: Each mapping consumes cluster memory
- Index performance: Large mapping definitions can slow down index operations
- Cluster stability: In extreme cases, mapping explosion can cause cluster instability
Note on strict mapping: If you set your index to use "dynamic": "strict", OpenSearch will reject any documents with unmapped fields, preventing mapping explosion but requiring you to explicitly define all object structures upfront. Despite this, the same index level mapping limits apply, so a very large explicit schema is still at risk of hitting this.
Flattened fields (OS flat object, ES flattened) solve this by treating the entire object as a single field, with all sub-properties indexed as keywords. This dramatically reduces the number of field mappings while still allowing you to search on object properties—and it works well even with strict mapping since you only need to define one flattened field mapping instead of many object field mappings. A wise Code Sloth takes the simplest, most pragmatic path after all 🦥
Let’s see how this works in practice.
The Basics: Simple Flattened Field Mapping
Let’s start with a simple example. We’ll use the ProductWithFlattenedAttribute class from our test suite to demonstrate how to create a flattened mapping and index documents with object attributes.
First, let’s look at the ProductWithFlattenedAttribute document structure:
public class ProductWithFlattenedAttribute implements IDocumentWithId {
private String id;
private String name;
private ProductAttribute attribute; // This will be mapped as a flattened field
// Constructors, getters, setters...
}
This document has three fields:
id: The product identifiername: The product nameattribute: An aggregated object containing product attributes (this will be mapped as a flattened field)
Now let’s look at the ProductAttribute record structure that represents the nested object:
public record ProductAttribute(
String color,
String size
) {
}
This record describes the color and size of a product. When nested inside ProductWithFlattenedAttribute, the attribute field will be mapped as a flattened field type.
Now let’s see how we can use this to define the flattened mapping in FlattenedIndexingTests.java:
@Test
public void flattenedMapping_IndexesObjectWithSubProperties() throws Exception {
// Create a test index with flattened mapping for the attribute field
try (OpenSearchTestIndex testIndex = fixture.createTestIndex(mapping ->
mapping.properties("attribute", Property.of(p -> p.flatObject(f -> f))))) {
// Create and index a product document with a strongly-typed record
// ProductAttribute: (color, size)
ProductWithFlattenedAttribute productDocument = new ProductWithFlattenedAttribute(
"1", "Product1", new ProductAttribute("red", "large"));
testIndex.indexDocuments(new ProductWithFlattenedAttribute[]{productDocument});
// Retrieve the document
GetResponse<ProductWithFlattenedAttribute> result = loggingOpenSearchClient.getClient().get(g -> g
.index(testIndex.getName())
.id(productDocument.getId()),
ProductWithFlattenedAttribute.class
);
// Verify the results
assertThat(result.found()).isTrue();
assertThat(result.source()).isNotNull();
assertThat(result.source().getName()).isEqualTo("Product1");
// Verify attribute was stored (it will be deserialized as a ProductAttribute record)
ProductAttribute storedAttribute = result.source().getAttribute();
assertThat(storedAttribute).isNotNull();
assertThat(storedAttribute.color()).isEqualTo("red");
assertThat(storedAttribute.size()).isEqualTo("large");
}
}
The key part here is the mapping definition: mapping.properties("attribute", Property.of(p -> p.flatObject(f -> f))). This tells OpenSearch to treat the attribute field as a flattened object, which means all properties within attribute (like color and size) will be indexed as keywords under the attribute field. Pretty straightforward, right?
When indexed, you can search on any of these properties using dotted notation: attribute.color, attribute.size, etc. Simple!
HTTP Commands for OpenSearch Dashboards
Here’s the index creation request that sets up the flattened field mapping:
PUT /flat_object_basic_indexing
{
"mappings": {
"properties": {
"attribute": {
"type": "flat_object"
}
}
},
"settings": {
"number_of_replicas": 0,
"number_of_shards": 1
}
}
Index Document Request:
POST /flat_object_basic_indexing/_bulk
{ "index" : { "_index" : "flat_object_basic_indexing", "_id" : "1" } }
{ "id" : "1", "name" : "Product1", "attribute" : { "color" : "red", "size" : "large" } }
GET Request: Retrieving document by ID
GET /flat_object_basic_indexing/_doc/1
GET Response:
{
"_index": "flat_object_basic_indexing",
"_id": "1",
"_version": 1,
"_seq_no": 0,
"_primary_term": 1,
"found": true,
"_source": {
"id": "1",
"name": "Product1",
"attribute": {
"color": "red",
"size": "large"
}
}
}
This test uses a GET request to retrieve the document by ID (not a search query). The document is retrieved directly and the flattened attribute structure is preserved when deserialized back into the ProductAttribute record.
What the Mapping Looks Like in OpenSearch
To retrieve the index mapping, use the following GET request:
GET /flat_object_basic_indexing/_mapping
GET Mapping Response:
When you query the index mapping in OpenSearch Dashboards, you’ll see the flattened field mapped as flat_object:
{
"d84606fe-7929-4641-bcb8-73d57843f73d": {
"mappings": {
"properties": {
"attribute": {
"type": "flat_object"
},
"id": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}
Note: Because we have dynamic mappings enabled (the default behavior), the other fields in the document (id and name) have automatically appeared in the mapping response. We’ve purposefully kept the mapping command simple to focus on the primary content of the post—the flattened field type. In production, you might want to explicitly define these mappings or use strict mapping to prevent automatic field creation.
Notice how the attribute field is simply defined as "type": "flat_object"—that’s all it takes! OpenSearch handles indexing all sub-properties as keywords automatically. You don’t need to define individual mappings for attribute.color or attribute.size; the flattened field type takes care of that for you.
Observing Keywords: How Flattened Fields Index Sub-Properties
Before we dive into searching, let’s verify exactly how flattened fields index sub-properties. Important caveat: Term vectors return misleading information for flattened fields!
Here’s what you need to know: flattened fields index sub-properties as unanalyzed keywords, meaning they preserve exact values including punctuation and capitalization. However, term vectors incorrectly suggest that tokenization occurs. Search behavior proves the true implementation—flattened fields store exact values without analysis.
Note: This term vector issue has been reported as a bug in GitHub issue #19864. Until this is fixed, rely on search behavior rather than term vectors to understand how flattened fields actually work.
The flattenedMapping_SubPropertiesIndexedAsKeywords test demonstrates this by showing that term vectors return incorrect tokenization results, which we’ll then disprove with search tests.
@Test
public void flattenedMapping_SubPropertiesIndexedAsKeywords() throws Exception {
// Create a test index with flattened mapping for the attribute field
try (OpenSearchTestIndex testIndex = fixture.createTestIndex(mapping ->
mapping.properties("attribute", Property.of(p -> p.flatObject(f -> f))))) {
// Create and index a product document with specific values that include capital letters and punctuation
// Note: Despite what term vectors show, flattened fields actually preserve exact values
ProductWithFlattenedAttribute productDocument = new ProductWithFlattenedAttribute(
"1", "Product1", new ProductAttribute("Red-Metal!", "Extra-Large"));
testIndex.indexDocuments(new ProductWithFlattenedAttribute[]{productDocument});
// This demonstrates that no tokens are created at this level
TermvectorsResponse topLevelResults = loggingOpenSearchClient.termvectors(t -> t
.index(testIndex.getName())
.id(productDocument.getId())
.fields("attribute")
);
// Verify that no tokens are produced at the top-level flattened field
// Flattened fields don't create tokens at the parent level, only at sub-property levels
assertThat(topLevelResults.found()).isTrue();
Map<String, TermVector> topLevelTermVectors = topLevelResults.termVectors();
// The term vectors map should be empty or contain no terms because flattened fields
// only index sub-properties, not the parent field itself
assertThat(topLevelTermVectors).isEmpty();
// Get term vectors for the color sub-property
// NOTE: Term vectors show INCORRECT results - they suggest tokenization happens,
// but flattened fields actually store unanalyzed keywords preserving exact values
TermvectorsResponse colorResult = loggingOpenSearchClient.termvectors(t -> t
.index(testIndex.getName())
.id(productDocument.getId())
.fields("attribute.color")
);
// WARNING: Term vectors show misleading tokenization results
// They incorrectly suggest "Red-Metal!" is tokenized to "red" and "metal"
// However, flattened fields are UNANALYZED keywords that preserve exact values
assertThat(colorResult.found()).isTrue();
Map<String, TermVector> colorTermVectors = colorResult.termVectors();
assertThat(colorTermVectors).hasSize(1);
TermVector colorTermVector = colorTermVectors.get("attribute.color");
assertThat(colorTermVector).isNotNull();
// Term vectors incorrectly show tokenization - "Red-Metal!" appears as "red" and "metal"
// BUT this is NOT how flattened fields actually work. See search tests for proof.
var colorTerms = colorTermVector.terms();
assertThat(colorTerms).hasSize(2);
assertThat(colorTerms).containsKey("red");
assertThat(colorTerms).containsKey("metal");
assertThat(colorTerms.get("red").termFreq()).isEqualTo(1);
assertThat(colorTerms.get("metal").termFreq()).isEqualTo(1);
// Get term vectors for the size sub-property
TermvectorsResponse sizeResult = loggingOpenSearchClient.termvectors(t -> t
.index(testIndex.getName())
.id(productDocument.getId())
.fields("attribute.size")
);
// WARNING: Term vectors show misleading tokenization results here too
assertThat(sizeResult.found()).isTrue();
Map<String, TermVector> sizeTermVectors = sizeResult.termVectors();
assertThat(sizeTermVectors).hasSize(1);
TermVector sizeTermVector = sizeTermVectors.get("attribute.size");
assertThat(sizeTermVector).isNotNull();
// Term vectors incorrectly show "Extra-Large" as "extra" and "large"
// BUT flattened fields actually preserve the exact value "Extra-Large" as-is
var sizeTerms = sizeTermVector.terms();
assertThat(sizeTerms).hasSize(2);
assertThat(sizeTerms).containsKey("extra");
assertThat(sizeTerms).containsKey("large");
assertThat(sizeTerms.get("extra").termFreq()).isEqualTo(1);
assertThat(sizeTerms.get("large").termFreq()).isEqualTo(1);
// IMPORTANT: Do not rely on term vectors to understand flattened field behavior.
// Term vectors return incorrect/ misleading information. Flattened fields are unanalyzed
// keywords that preserve exact values. Search tests prove this by showing that:
// - Exact matches work: "Red-Metal!" matches "Red-Metal!"
// - Token searches fail: "red" does NOT match "Red-Metal!"
}
}
Note: The term vectors responses show misleading tokenization results. Despite what they display (showing “red” and “metal” tokens from “Red-Metal!”, and “extra” and “large” from “Extra-Large”), flattened fields preserve exact values as unanalyzed keywords. See the search tests in the next section for proof of the actual behavior.
This test demonstrates that term vectors return incorrect results for flattened fields:
-
Top-level flattened fields produce no tokens: Querying
attributedirectly returns an empty term vector map. This part is correct—flattened fields don’t create tokens at the parent level. -
Term vectors incorrectly show tokenization: When we query
attribute.color, term vectors show two tokens:"red"and"metal", suggesting that"Red-Metal!"was tokenized. This is misleading! The same happens forattribute.size—term vectors show"extra"and"large"from"Extra-Large". -
Search behavior proves term vectors wrong: As we’ll see in the next section, search tests prove that flattened fields actually preserve exact values. Searching for
"red"does NOT match"Red-Metal!", proving that no tokenization occurs.
Key takeaway: Term vectors are unreliable for understanding flattened field behavior. Flattened fields index sub-properties as unanalyzed keywords that preserve exact values, including punctuation and capitalization. This term vector issue has been reported in GitHub issue #19864.
HTTP Commands for OpenSearch Dashboards
Here’s the index creation request that sets up the flattened field mapping:
PUT /flat_object_term_vectors
{
"mappings": {
"properties": {
"attribute": {
"type": "flat_object"
}
}
},
"settings": {
"number_of_replicas": 0,
"number_of_shards": 1
}
}
Index Document Request:
POST /flat_object_term_vectors/_bulk
{ "index" : { "_index" : "flat_object_term_vectors", "_id" : "1" } }
{ "id" : "1", "name" : "Product1", "attribute" : { "color" : "Red-Metal!", "size" : "Extra-Large!" } }
GET Mapping Request:
GET /flat_object_term_vectors/_mapping
GET Mapping Response:
{
"flat_object_term_vectors": {
"mappings": {
"properties": {
"attribute": {
"type": "flat_object"
},
"id": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}
Term Vectors Request 1: Querying the top-level flattened field
POST /flat_object_term_vectors/_termvectors/1
{
"fields": ["attribute"]
}
Term Vectors Response 1:
{
"found": true,
"_id": "1",
"_index": "flat_object_term_vectors",
"term_vectors": {},
"took": 21,
"_version": 1
}
Term Vectors Request 2: Querying attribute.color sub-property
POST /flat_object_term_vectors/_termvectors/1
{
"fields": ["attribute.color"]
}
Term Vectors Response 2:
{
"found": true,
"_id": "1",
"_index": "flat_object_term_vectors",
"term_vectors": {
"attribute.color": {
"field_statistics": {
"doc_count": 0,
"sum_doc_freq": 0,
"sum_ttf": 0
},
"terms": {
"red": {
"term_freq": 1,
"tokens": [
{
"end_offset": 3,
"position": 0,
"start_offset": 0
}
]
},
"metal": {
"term_freq": 1,
"tokens": [
{
"end_offset": 9,
"position": 1,
"start_offset": 4
}
]
}
}
}
},
"took": 5,
"_version": 1
}
Term Vectors Request 3: Querying attribute.size sub-property
POST /flat_object_term_vectors/_termvectors/1
{
"fields": ["attribute.size"]
}
Term Vectors Response 3:
{
"found": true,
"_id": "1",
"_index": "flat_object_term_vectors",
"term_vectors": {
"attribute.size": {
"field_statistics": {
"doc_count": 0,
"sum_doc_freq": 0,
"sum_ttf": 0
},
"terms": {
"extra": {
"term_freq": 1,
"tokens": [
{
"end_offset": 5,
"position": 0,
"start_offset": 0
}
]
},
"large": {
"term_freq": 1,
"tokens": [
{
"end_offset": 11,
"position": 1,
"start_offset": 6
}
]
}
}
}
},
"took": 0,
"_version": 1
}
Searching on Flattened Fields
Once you’ve indexed documents with flattened fields, searching works much like searching on any other field—you just use dotted notation to access nested properties. That’s it!
The flat object field type supports the following queries:
- Term
- Terms
- Terms set
- Prefix
- Range
- Match
- Multi-match
- Query string
- Simple query string
- Exists
- Wildcard
Let’s break down how we search on flattened fields using examples from FlattenedSearchingTests.java:
@Test
public void flattenedMapping_SingleField_CanSearchByDottedNotation() throws Exception {
// Create a test index with flattened mapping for the attribute field
try (OpenSearchTestIndex testIndex = fixture.createTestIndex(mapping ->
mapping.properties("attribute", Property.of(p -> p.flatObject(f -> f))))) {
// Create documents with attributes containing multiple properties using strongly-typed records
// ProductAttribute: (color, size)
ProductWithFlattenedAttribute[] products = new ProductWithFlattenedAttribute[]{
new ProductWithFlattenedAttribute("1", "Product1", new ProductAttribute("red", "large")),
new ProductWithFlattenedAttribute("2", "Product2", new ProductAttribute("blue", "medium")),
new ProductWithFlattenedAttribute("3", "Product3", new ProductAttribute("red", "small"))
};
testIndex.indexDocuments(products);
// Search by attribute.color using term query
SearchResponse<ProductWithFlattenedAttribute> result = loggingOpenSearchClient.search(s -> s
.index(testIndex.getName())
.query(q -> q
.term(t -> t
.field("attribute.color")
.value(FieldValue.of("red"))
)
),
ProductWithFlattenedAttribute.class
);
// With flatObject type, dotted notation queries work correctly
assertThat(result.hits().total().value()).isEqualTo(2);
assertThat(result.hits().hits().stream()
.map(h -> h.source().getId())
.sorted())
.containsExactly("1", "3"); // Product1 and Product3 have red color
// Search by attribute.size using dotted notation
SearchResponse<ProductWithFlattenedAttribute> sizeResult = loggingOpenSearchClient.search(s -> s
.index(testIndex.getName())
.query(q -> q
.term(t -> t
.field("attribute.size")
.value(FieldValue.of("large"))
)
),
ProductWithFlattenedAttribute.class
);
// With flatObject type, size search works correctly
assertThat(sizeResult.hits().total().value()).isEqualTo(1);
assertThat(sizeResult.hits().hits().stream()
.map(h -> h.source().getId())
.sorted())
.containsExactly("1"); // Only Product1 has large size
}
}
See how simple that is? You can search on flattened field properties using term queries with dotted notation. We search for attribute.color="red" and attribute.size="large", and OpenSearch correctly finds the documents. The dotted notation (like attribute.color) gives you direct access to nested properties within the flattened field—no magic required!
HTTP Commands for OpenSearch Dashboards
Here’s the index creation request:
PUT /flat_object_basic_search
{
"mappings": {
"properties": {
"attribute": {
"type": "flat_object"
}
}
},
"settings": {
"number_of_replicas": 0,
"number_of_shards": 1
}
}
Index Documents Request:
POST /flat_object_basic_search/_bulk
{ "index" : { "_index" : "flat_object_basic_search", "_id" : "1" } }
{ "id" : "1", "name" : "Product1", "attribute" : { "color" : "red", "size" : "large" } }
{ "index" : { "_index" : "flat_object_basic_search", "_id" : "2" } }
{ "id" : "2", "name" : "Product2", "attribute" : { "color" : "blue", "size" : "medium" } }
{ "index" : { "_index" : "flat_object_basic_search", "_id" : "3" } }
{ "id" : "3", "name" : "Product3", "attribute" : { "color" : "red", "size" : "small" } }
Search Request 1: Finding products by color
POST flat_object_basic_search/_search
{
"query": {
"term": {
"attribute.color": {
"value": "red"
}
}
}
}
Search Response 1:
{
"took": 10,
"timed_out": false,
"_shards": {
"failed": 0,
"skipped": 0,
"successful": 1,
"total": 1
},
"hits": {
"hits": [
{
"_id": "1",
"_index": "flat_object_basic_search",
"_score": 0.5908618,
"_source": {
"id": "1",
"name": "Product1",
"attribute": {
"color": "red",
"size": "large"
}
}
},
{
"_id": "3",
"_index": "flat_object_basic_search",
"_score": 0.5908618,
"_source": {
"id": "3",
"name": "Product3",
"attribute": {
"color": "red",
"size": "small"
}
}
}
],
"max_score": 0.5908617973327637,
"total": {
"relation": "eq",
"value": 2
}
}
}
Search Request 2: Finding products by size
POST flat_object_basic_search/_search
{
"query": {
"term": {
"attribute.size": {
"value": "large"
}
}
}
}
Search Response 2:
{
"took": 2,
"timed_out": false,
"_shards": {
"failed": 0,
"skipped": 0,
"successful": 1,
"total": 1
},
"hits": {
"hits": [
{
"_id": "1",
"_index": "flat_object_exact_match",
"_score": 1.2330425,
"_source": {
"id": "1",
"name": "Product1",
"attribute": {
"color": "red",
"size": "large"
}
}
}
],
"max_score": 1.2330424785614014,
"total": {
"relation": "eq",
"value": 1
}
}
}
Proving Unanalyzed Behavior: Exact Matches Only
Remember how term vectors incorrectly showed tokenization? Let’s prove that flattened fields are actually unanalyzed by testing search behavior with punctuation. The flattenedMapping_DoesNotAnalyzeInputs test demonstrates that flattened mapping does not analyze inputs:
@Test
public void flattenedMapping_DoesNotAnalyzeInputs() throws Exception {
// Create a test index with flattened mapping for the attribute field
try (OpenSearchTestIndex testIndex = fixture.createTestIndex(mapping ->
mapping.properties("attribute", Property.of(p -> p.flatObject(f -> f))))) {
// Create and index a product document with punctuation in the attribute
// Flattened mapping does not analyze inputs, so these exact values are preserved as-is
ProductWithFlattenedAttribute productDocument = new ProductWithFlattenedAttribute(
"1", "Product1", new ProductAttribute("Red-Metal!", "Extra-Large!"));
testIndex.indexDocuments(new ProductWithFlattenedAttribute[]{productDocument});
// Search for the EXACT term with punctuation - this SHOULD work because flattened
// mapping does not analyze inputs, so values are preserved exactly as provided
SearchResponse<ProductWithFlattenedAttribute> exactMatchSearch = loggingOpenSearchClient.search(s -> s
.index(testIndex.getName())
.query(q -> q
.term(t -> t
.field("attribute.color")
.value(FieldValue.of("Red-Metal!"))
)
),
ProductWithFlattenedAttribute.class
);
// Verify that searching for the exact term with punctuation DOES work
// Flattened fields preserve exact values, so "Red-Metal!" matches "Red-Metal!"
assertThat(exactMatchSearch.hits().total().value()).isEqualTo(1);
assertThat(exactMatchSearch.hits().hits().get(0).source().getId()).isEqualTo("1");
// Similarly, searching for exact "Extra-Large!" should also work
SearchResponse<ProductWithFlattenedAttribute> exactSizeSearch = loggingOpenSearchClient.search(s -> s
.index(testIndex.getName())
.query(q -> q
.term(t -> t
.field("attribute.size")
.value(FieldValue.of("Extra-Large!"))
)
),
ProductWithFlattenedAttribute.class
);
// Verify that searching for exact term DOES work
assertThat(exactSizeSearch.hits().total().value()).isEqualTo(1);
assertThat(exactSizeSearch.hits().hits().get(0).source().getId()).isEqualTo("1");
// Now disprove tokenization by searching for individual tokens from term vector output
// Term vectors incorrectly show "Red-Metal!" as "red" and "metal", but searches prove this is wrong
SearchResponse<ProductWithFlattenedAttribute> tokenRedSearch = loggingOpenSearchClient.search(s -> s
.index(testIndex.getName())
.query(q -> q
.term(t -> t
.field("attribute.color")
.value(FieldValue.of("red"))
)
),
ProductWithFlattenedAttribute.class
);
// Searching for "red" should return NO results - proving no tokenization occurs
// If tokenization happened, "red" would match "Red-Metal!", but it doesn't
assertThat(tokenRedSearch.hits().total().value()).isEqualTo(0);
assertThat(tokenRedSearch.hits().hits()).isEmpty();
// Search for "metal" token - should also return NO results
SearchResponse<ProductWithFlattenedAttribute> tokenMetalSearch = loggingOpenSearchClient.search(s -> s
.index(testIndex.getName())
.query(q -> q
.term(t -> t
.field("attribute.color")
.value(FieldValue.of("metal"))
)
),
ProductWithFlattenedAttribute.class
);
// Searching for "metal" should return NO results - proving no tokenization
assertThat(tokenMetalSearch.hits().total().value()).isEqualTo(0);
assertThat(tokenMetalSearch.hits().hits()).isEmpty();
// Search for "extra" token from size - should return NO results
SearchResponse<ProductWithFlattenedAttribute> tokenExtraSearch = loggingOpenSearchClient.search(s -> s
.index(testIndex.getName())
.query(q -> q
.term(t -> t
.field("attribute.size")
.value(FieldValue.of("extra"))
)
),
ProductWithFlattenedAttribute.class
);
// Searching for "extra" should return NO results - proving no tokenization
assertThat(tokenExtraSearch.hits().total().value()).isEqualTo(0);
assertThat(tokenExtraSearch.hits().hits()).isEmpty();
// Search for "large" token from size - should return NO results
SearchResponse<ProductWithFlattenedAttribute> tokenLargeSearch = loggingOpenSearchClient.search(s -> s
.index(testIndex.getName())
.query(q -> q
.term(t -> t
.field("attribute.size")
.value(FieldValue.of("large"))
)
),
ProductWithFlattenedAttribute.class
);
// Searching for "large" should return NO results - proving no tokenization
assertThat(tokenLargeSearch.hits().total().value()).isEqualTo(0);
assertThat(tokenLargeSearch.hits().hits()).isEmpty();
// This demonstrates that flattened fields are UNANALYZED keywords that preserve exact values.
// Term vectors incorrectly show tokenization, but search behavior proves values are stored exactly.
// You must search using the exact value as indexed, including punctuation and capitalization.
}
}
This test proves that flattened mapping does not analyze inputs:
-
Exact matches work: Searching for
"Red-Metal!"finds the document indexed with"Red-Metal!". The exact value with punctuation and capitalization is preserved because no analysis occurs. Same for"Extra-Large!"—it matches exactly as stored. -
Token searches fail: Searching for
"red","metal","extra", or"large"(the tokens that term vectors incorrectly showed) all return zero results. This proves that no analysis or tokenization occurred during indexing. -
Case and punctuation matter: Since flattened mapping does not analyze inputs, you must search using the exact value as indexed—including capitalization and punctuation.
"red"does not match"Red-Metal!"because they are different exact values.
The verdict: Term vectors are misleading. Flattened mapping does not analyze inputs—sub-properties are indexed as unanalyzed keywords that preserve exact values. Always use exact matches when searching flattened field sub-properties. This term vector bug has been reported in GitHub issue #19864.
HTTP Commands for OpenSearch Dashboards
Here’s the index creation request (same as previous examples):
PUT /flat_object_exact_match
{
"mappings": {
"properties": {
"attribute": {
"type": "flat_object"
}
}
},
"settings": {
"number_of_replicas": 0,
"number_of_shards": 1
}
}
Index Document Request:
POST /flat_object_exact_match/_bulk
{ "index" : { "_index" : "flat_object_exact_match", "_id" : "1" } }
{ "id" : "1", "name" : "Product1", "attribute" : { "color" : "Red-Metal!", "size" : "Extra-Large!" } }
Exact Match Search Request 1: Searching for exact color value with punctuation
POST flat_object_exact_match/_search
{
"query": {
"term": {
"attribute.color": {
"value": "Red-Metal!"
}
}
}
}
Search Response (matches):
{
"took": 9,
"timed_out": false,
"_shards": {
"failed": 0,
"skipped": 0,
"successful": 1,
"total": 1
},
"hits": {
"hits": [
{
"_id": "1",
"_index": "flat_object_exact_match",
"_score": 0.3616575,
"_source": {
"id": "1",
"name": "Product1",
"attribute": {
"color": "Red-Metal!",
"size": "Extra-Large!"
}
}
}
],
"max_score": 0.3616575002670288,
"total": {
"relation": "eq",
"value": 1
}
}
}
Exact Match Search Request 2: Searching for exact size value with punctuation
POST flat_object_exact_match/_search
{
"query": {
"term": {
"attribute.size": {
"value": "Extra-Large!"
}
}
}
}
Search Response (matches):
{
"took": 2,
"timed_out": false,
"_shards": {
"failed": 0,
"skipped": 0,
"successful": 1,
"total": 1
},
"hits": {
"hits": [
{
"_id": "1",
"_index": "flat_object_exact_match",
"_score": 0.3616575,
"_source": {
"id": "1",
"name": "Product1",
"attribute": {
"color": "Red-Metal!",
"size": "Extra-Large!"
}
}
}
],
"max_score": 0.3616575002670288,
"total": {
"relation": "eq",
"value": 1
}
}
}
Token Search Request 1: Searching for token “red” (should fail)
POST flat_object_exact_match/_search
{
"query": {
"term": {
"attribute.color": {
"value": "red"
}
}
}
}
Search Response (no matches – proves no tokenization):
{
"took": 2,
"timed_out": false,
"_shards": {
"failed": 0,
"skipped": 0,
"successful": 1,
"total": 1
},
"hits": {
"hits": [],
"total": {
"relation": "eq",
"value": 0
}
}
}
Similarly, searching for other tokens like "metal", "extra", or "large" all return zero results, proving that flattened fields store exact values without tokenization or analysis.
Matching Multiple Properties from the Same Flattened Field
Here’s something important to understand: you can match multiple properties from within a single flattened object. This is great news! However, this is different from flattened arrays (which we’ll discuss later), where matching multiple properties from the same object in an array doesn’t work reliably.
Let’s look at a test that demonstrates matching multiple properties from the same flattened field, using the flattenedMapping_SingleField_CanMatchMultiplePropertiesFromSameObject test:
@Test
public void flattenedMapping_SingleField_CanMatchMultiplePropertiesFromSameObject() throws Exception {
// Create a test index with flattened mapping for the attribute field
try (OpenSearchTestIndex testIndex = fixture.createTestIndex(mapping ->
mapping.properties("attribute", Property.of(p -> p.flatObject(f -> f))))) {
// Create documents with attributes containing multiple properties
// ProductAttribute: (color, size)
ProductWithFlattenedAttribute[] products = new ProductWithFlattenedAttribute[]{
new ProductWithFlattenedAttribute("1", "Product1", new ProductAttribute("red", "large")),
new ProductWithFlattenedAttribute("2", "Product2", new ProductAttribute("blue", "medium")),
new ProductWithFlattenedAttribute("3", "Product3", new ProductAttribute("red", "small")),
new ProductWithFlattenedAttribute("4", "Product4", new ProductAttribute("red", "large"))
};
testIndex.indexDocuments(products);
// Search for products where attribute.color="red" AND attribute.size="large"
// This should work because we're matching multiple properties from the same flattened object
SearchResponse<ProductWithFlattenedAttribute> result = loggingOpenSearchClient.search(s -> s
.index(testIndex.getName())
.query(q -> q
.bool(b -> b
.must(m -> m
.term(t -> t
.field("attribute.color")
.value(FieldValue.of("red"))
)
)
.must(m -> m
.term(t -> t
.field("attribute.size")
.value(FieldValue.of("large"))
)
)
)
),
ProductWithFlattenedAttribute.class
);
// With flatObject type, matching multiple properties from the same object works correctly
// Should match Product1 (id="1") and Product4 (id="4") - both red AND large
assertThat(result.hits().total().value()).isEqualTo(2);
assertThat(result.hits().hits().stream()
.map(h -> h.source().getId())
.sorted())
.containsExactly("1", "4"); // Product1 and Product4 match: red AND large
// Verify the actual documents match our expectations
assertThat(result.hits().hits().stream()
.anyMatch(h -> h.source().getId().equals("1") && h.source().getName().equals("Product1")))
.isTrue();
assertThat(result.hits().hits().stream()
.anyMatch(h -> h.source().getId().equals("4") && h.source().getName().equals("Product4")))
.isTrue();
// Test with different combination: color="red" AND size="small"
SearchResponse<ProductWithFlattenedAttribute> smallResult = loggingOpenSearchClient.search(s -> s
.index(testIndex.getName())
.query(q -> q
.bool(b -> b
.must(m -> m
.term(t -> t
.field("attribute.color")
.value(FieldValue.of("red"))
)
)
.must(m -> m
.term(t -> t
.field("attribute.size")
.value(FieldValue.of("small"))
)
)
)
),
ProductWithFlattenedAttribute.class
);
// Should match only Product3 (id="3") - red AND small
assertThat(smallResult.hits().total().value()).isEqualTo(1);
assertThat(smallResult.hits().hits().get(0).source().getId()).isEqualTo("3");
assertThat(smallResult.hits().hits().get(0).source().getName()).isEqualTo("Product3");
}
}
This test demonstrates that when you have a single flattened object (not an array), you can successfully search for multiple properties within that same object using boolean queries. Pretty powerful, right? The query for attribute.color="red" AND attribute.size="large" correctly finds documents where both conditions are true within the same flattened object.
This works because flattened fields preserve the relationship between properties when they’re in a single object. However, as we’ll see later, this behavior changes when you’re working with arrays of flattened objects. But don’t worry—we’ll walk through that step by step.
HTTP Commands for OpenSearch Dashboards
Here’s the index creation request:
PUT /flat_object_multiple_properties
{
"mappings": {
"properties": {
"attribute": {
"type": "flat_object"
}
}
},
"settings": {
"number_of_replicas": 0,
"number_of_shards": 1
}
}
Index Documents Request:
POST /flat_object_multiple_properties/_bulk
{ "index" : { "_index" : "flat_object_multiple_properties", "_id" : "1" } }
{ "id" : "1", "name" : "Product1", "attribute" : { "color" : "red", "size" : "large" } }
{ "index" : { "_index" : "flat_object_multiple_properties", "_id" : "2" } }
{ "id" : "2", "name" : "Product2", "attribute" : { "color" : "blue", "size" : "medium" } }
{ "index" : { "_index" : "flat_object_multiple_properties", "_id" : "3" } }
{ "id" : "3", "name" : "Product3", "attribute" : { "color" : "red", "size" : "small" } }
{ "index" : { "_index" : "flat_object_multiple_properties", "_id" : "4" } }
{ "id" : "4", "name" : "Product4", "attribute" : { "color" : "red", "size" : "large" } }
Search Request 1: Matching multiple properties (color=“red” AND size=“large”)
POST flat_object_multiple_properties/_search
{
"query": {
"bool": {
"must": [
{
"term": {
"attribute.color": {
"value": "red"
}
}
},
{
"term": {
"attribute.size": {
"value": "large"
}
}
}
]
}
}
}
Search Response 1:
{
"took": 16,
"timed_out": false,
"_shards": {
"failed": 0,
"skipped": 0,
"successful": 1,
"total": 1
},
"hits": {
"hits": [
{
"_id": "1",
"_index": "flat_object_multiple_properties",
"_score": 1.3197765,
"_source": {
"id": "1",
"name": "Product1",
"attribute": {
"color": "red",
"size": "large"
}
}
},
{
"_id": "4",
"_index": "flat_object_multiple_properties",
"_score": 1.3197765,
"_source": {
"id": "4",
"name": "Product4",
"attribute": {
"color": "red",
"size": "large"
}
}
}
],
"max_score": 1.3197765350341797,
"total": {
"relation": "eq",
"value": 2
}
}
}
Search Request 2: Different combination (color=“red” AND size=“small”)
POST flat_object_multiple_properties/_search
{
"query": {
"bool": {
"must": [
{
"term": {
"attribute.color": {
"value": "red"
}
}
},
{
"term": {
"attribute.size": {
"value": "small"
}
}
}
]
}
}
}
Search Response 2:
{
"took": 5,
"timed_out": false,
"_shards": {
"failed": 0,
"skipped": 0,
"successful": 1,
"total": 1
},
"hits": {
"hits": [
{
"_id": "3",
"_index": "flat_object_multiple_properties",
"_score": 1.9619575,
"_source": {
"id": "3",
"name": "Product3",
"attribute": {
"color": "red",
"size": "small"
}
}
}
],
"max_score": 1.9619574546813965,
"total": {
"relation": "eq",
"value": 1
}
}
}
Multiple Flattened Fields in a Single Document
Good news: you’re not limited to a single flattened field per document! You can have multiple flattened fields, and you can search across them independently or together. Let’s see how this works with the ProductWithTwoFlattenedAttributes class:
public class ProductWithTwoFlattenedAttributes implements IDocumentWithId {
private String id;
private String name;
private ProductAttribute primaryAttribute; // Flattened field for primary attribute information
private ProductSpecification secondaryAttribute; // Flattened field for secondary attribute information (different DTO type)
// ... constructor and getters/setters
}
Notice that each flattened field can use any DTO type you want—they can be the same type or different types. In this example, primaryAttribute uses ProductAttribute:
public record ProductAttribute(
String color,
String size
) {
}
While secondaryAttribute uses ProductSpecification:
public record ProductSpecification(
String brand,
String category
) {
}
You can use the same DTO type for multiple flattened fields, or different types—it’s entirely up to you! Each flattened field can have its own structure with whatever properties you need.
Here’s how we create an index with multiple flattened fields and search across them:
@Test
public void flattenedMapping_MultipleFlattenedFields_CanMatchFromBothFields() throws Exception {
// Create a test index with multiple flattened fields: primaryAttribute and secondaryAttribute
try (OpenSearchTestIndex testIndex = fixture.createTestIndex(mapping -> {
mapping.properties("primaryAttribute", Property.of(p -> p.flatObject(f -> f)));
mapping.properties("secondaryAttribute", Property.of(p -> p.flatObject(f -> f)));
})) {
// Create documents with both primary and secondary attributes using different DTO types
// Primary uses ProductAttribute: (color, size)
// Secondary uses ProductSpecification: (brand, category) - different DTO demonstrates flexibility
ProductWithTwoFlattenedAttributes[] products = new ProductWithTwoFlattenedAttributes[]{
new ProductWithTwoFlattenedAttributes("1", "Product1",
new ProductAttribute("red", "large"),
new ProductSpecification("BrandA", "Electronics")),
new ProductWithTwoFlattenedAttributes("2", "Product2",
new ProductAttribute("red", "medium"),
new ProductSpecification("BrandB", "Clothing")),
new ProductWithTwoFlattenedAttributes("3", "Product3",
new ProductAttribute("blue", "small"),
new ProductSpecification("BrandA", "Electronics"))
};
testIndex.indexDocuments(products);
// Search for products with primaryAttribute.color="red" AND secondaryAttribute.brand="BrandA"
// This should work because we're matching across different flattened fields with different DTO types
SearchResponse<ProductWithTwoFlattenedAttributes> result = loggingOpenSearchClient.search(s -> s
.index(testIndex.getName())
.query(q -> q
.bool(b -> b
.must(m -> m
.term(t -> t
.field("primaryAttribute.color")
.value(FieldValue.of("red"))
)
)
.must(m -> m
.term(t -> t
.field("secondaryAttribute.brand")
.value(FieldValue.of("BrandA"))
)
)
)
),
ProductWithTwoFlattenedAttributes.class
);
// With flatObject type, matching across multiple flattened fields works correctly
// Note: primaryAttribute uses ProductAttribute (color, size) while secondaryAttribute
// uses ProductSpecification (brand, category) - demonstrating different DTO types work
assertThat(result.hits().total().value()).isEqualTo(1);
assertThat(result.hits().hits().stream()
.map(h -> h.source().getId()))
.containsExactly("1"); // Product1 matches: primaryAttribute.color="red" AND secondaryAttribute.brand="BrandA"
// Search for primaryAttribute.color="red" alone - should match Product1 and Product2
SearchResponse<ProductWithTwoFlattenedAttributes> primaryResult = loggingOpenSearchClient.search(s -> s
.index(testIndex.getName())
.query(q -> q
.term(t -> t
.field("primaryAttribute.color")
.value(FieldValue.of("red"))
)
),
ProductWithTwoFlattenedAttributes.class
);
assertThat(primaryResult.hits().total().value()).isEqualTo(2);
assertThat(primaryResult.hits().hits().stream()
.map(h -> h.source().getId()))
.containsExactlyInAnyOrder("1", "2");
// Search for secondaryAttribute.category="Electronics" alone - should match Product1 and Product3
SearchResponse<ProductWithTwoFlattenedAttributes> secondaryResult = loggingOpenSearchClient.search(s -> s
.index(testIndex.getName())
.query(q -> q
.term(t -> t
.field("secondaryAttribute.category")
.value(FieldValue.of("Electronics"))
)
),
ProductWithTwoFlattenedAttributes.class
);
assertThat(secondaryResult.hits().total().value()).isEqualTo(2);
assertThat(secondaryResult.hits().hits().stream()
.map(h -> h.source().getId()))
.containsExactlyInAnyOrder("1", "3");
}
}
This example demonstrates something important: you CAN match multiple values across different flattened fields in the same document, and each flattened field can use any DTO type you want (same or different). When you search for primaryAttribute.color="red" AND secondaryAttribute.brand="BrandA", OpenSearch correctly finds documents where both conditions are true, even though they’re in different flattened fields. In this example, primaryAttribute uses ProductAttribute (with color and size), while secondaryAttribute uses ProductSpecification (with brand and category). You’re free to use the same DTO type for all flattened fields, or different types—whatever fits your data model!
HTTP Commands for OpenSearch Dashboards
Here’s the index creation request with multiple flattened fields:
PUT /flat_object_multiple_fields
{
"mappings": {
"properties": {
"primaryAttribute": {
"type": "flat_object"
},
"secondaryAttribute": {
"type": "flat_object"
}
}
},
"settings": {
"number_of_replicas": 0,
"number_of_shards": 1
}
}
Index Documents Request:
POST /flat_object_multiple_fields/_bulk
{ "index" : { "_index" : "flat_object_multiple_fields", "_id" : "1" } }
{ "id" : "1", "name" : "Product1", "primaryAttribute" : { "color" : "red", "size" : "large" }, "secondaryAttribute" : { "brand" : "BrandA", "category" : "Electronics" } }
{ "index" : { "_index" : "flat_object_multiple_fields", "_id" : "2" } }
{ "id" : "2", "name" : "Product2", "primaryAttribute" : { "color" : "red", "size" : "medium" }, "secondaryAttribute" : { "brand" : "BrandB", "category" : "Clothing" } }
{ "index" : { "_index" : "flat_object_multiple_fields", "_id" : "3" } }
{ "id" : "3", "name" : "Product3", "primaryAttribute" : { "color" : "blue", "size" : "small" }, "secondaryAttribute" : { "brand" : "BrandA", "category" : "Electronics" } }
Search Request 1: Matching across different flattened fields (primaryAttribute.color=“red” AND secondaryAttribute.brand=“BrandA”)
POST flat_object_multiple_fields/_search
{
"query": {
"bool": {
"must": [
{
"term": {
"primaryAttribute.color": {
"value": "red"
}
}
},
{
"term": {
"secondaryAttribute.brand": {
"value": "BrandA"
}
}
}
]
}
}
}
Search Response 1:
{
"took": 16,
"timed_out": false,
"_shards": {
"failed": 0,
"skipped": 0,
"successful": 1,
"total": 1
},
"hits": {
"hits": [
{
"_id": "1",
"_index": "flat_object_multiple_fields",
"_score": 1.1817236,
"_source": {
"id": "1",
"name": "Product1",
"primaryAttribute": {
"color": "red",
"size": "large"
},
"secondaryAttribute": {
"brand": "BrandA",
"category": "Electronics"
}
}
}
],
"max_score": 1.1817235946655273,
"total": {
"relation": "eq",
"value": 1
}
}
}
Search Request 2: Searching primaryAttribute alone
POST flat_object_multiple_fields/_search
{
"query": {
"term": {
"primaryAttribute.color": {
"value": "red"
}
}
}
}
Search Response 2:
{
"took": 2,
"timed_out": false,
"_shards": {
"failed": 0,
"skipped": 0,
"successful": 1,
"total": 1
},
"hits": {
"hits": [
{
"_id": "1",
"_index": "flat_object_multiple_fields",
"_score": 0.5908618,
"_source": {
"id": "1",
"name": "Product1",
"primaryAttribute": {
"color": "red",
"size": "large"
},
"secondaryAttribute": {
"brand": "BrandA",
"category": "Electronics"
}
}
},
{
"_id": "2",
"_index": "flat_object_multiple_fields",
"_score": 0.5908618,
"_source": {
"id": "2",
"name": "Product2",
"primaryAttribute": {
"color": "red",
"size": "medium"
},
"secondaryAttribute": {
"brand": "BrandB",
"category": "Clothing"
}
}
}
],
"max_score": 0.5908617973327637,
"total": {
"relation": "eq",
"value": 2
}
}
}
Search Request 3: Searching secondaryAttribute alone
POST flat_object_multiple_fields/_search
{
"query": {
"term": {
"secondaryAttribute.category": {
"value": "Electronics"
}
}
}
}
Search Response 3:
{
"took": 5,
"timed_out": false,
"_shards": {
"failed": 0,
"skipped": 0,
"successful": 1,
"total": 1
},
"hits": {
"hits": [
{
"_id": "1",
"_index": "flat_object_multiple_fields",
"_score": 0.5908618,
"_source": {
"id": "1",
"name": "Product1",
"primaryAttribute": {
"color": "red",
"size": "large"
},
"secondaryAttribute": {
"brand": "BrandA",
"category": "Electronics"
}
}
},
{
"_id": "3",
"_index": "flat_object_multiple_fields",
"_score": 0.5908618,
"_source": {
"id": "3",
"name": "Product3",
"primaryAttribute": {
"color": "blue",
"size": "small"
},
"secondaryAttribute": {
"brand": "BrandA",
"category": "Electronics"
}
}
}
],
"max_score": 0.5908617973327637,
"total": {
"relation": "eq",
"value": 2
}
}
}
Nested Flattened Fields: Multi-Level Dotted Notation
Flattened fields can also contain nested complex types, requiring multiple levels of dotted notation to access leaf properties. Don’t let that scare you—it’s actually quite straightforward! This is super useful when you have hierarchical data structures within a flattened field.
The ProductWithNestedFlattened document class demonstrates this pattern:
public class ProductWithNestedFlattened implements IDocumentWithId {
private String id;
private String name;
private ProductDetails details; // Flattened field containing nested ProductAttribute
// ... constructor and getters/setters
}
The ProductDetails record contains a nested ProductAttribute:
public record ProductDetails(
ProductAttribute attribute, // Nested ProductAttribute (color, size)
String description
) {
}
public record ProductAttribute(
String color,
String size
) {
}
Here’s how we search on nested flattened fields using multi-level dotted notation, from the flattenedMapping_NestedFlattenedField_CanSearchByNestedDottedNotation test:
@Test
public void flattenedMapping_NestedFlattenedField_CanSearchByNestedDottedNotation() throws Exception {
// Create a test index with flattened mapping for the details field
try (OpenSearchTestIndex testIndex = fixture.createTestIndex(mapping ->
mapping.properties("details", Property.of(p -> p.flatObject(f -> f))))) {
// Create documents with nested attribute structures
// ProductDetails: (attribute: ProductAttribute, description: String)
// ProductAttribute: (color, size)
ProductWithNestedFlattened[] products = new ProductWithNestedFlattened[]{
new ProductWithNestedFlattened("1", "Product1",
new ProductDetails(
new ProductAttribute("red", "large"),
"Premium quality product")),
new ProductWithNestedFlattened("2", "Product2",
new ProductDetails(
new ProductAttribute("blue", "medium"),
"Standard quality product")),
new ProductWithNestedFlattened("3", "Product3",
new ProductDetails(
new ProductAttribute("red", "small"),
"Compact design"))
};
testIndex.indexDocuments(products);
// Search using multi-level dotted notation: details.attribute.color
SearchResponse<ProductWithNestedFlattened> result = loggingOpenSearchClient.search(s -> s
.index(testIndex.getName())
.query(q -> q
.term(t -> t
.field("details.attribute.color")
.value(FieldValue.of("red"))
)
),
ProductWithNestedFlattened.class
);
// With flatObject type, nested dotted notation queries work correctly
assertThat(result.hits().total().value()).isEqualTo(2);
assertThat(result.hits().hits().stream()
.map(h -> h.source().getId())
.sorted())
.containsExactly("1", "3"); // Product1 and Product3 have red color in nested attribute
// Search using another nested property: details.attribute.size
SearchResponse<ProductWithNestedFlattened> sizeResult = loggingOpenSearchClient.search(s -> s
.index(testIndex.getName())
.query(q -> q
.term(t -> t
.field("details.attribute.size")
.value(FieldValue.of("large"))
)
),
ProductWithNestedFlattened.class
);
assertThat(sizeResult.hits().total().value()).isEqualTo(1);
assertThat(sizeResult.hits().hits().get(0).source().getId()).isEqualTo("1");
// Search using a direct property of details: details.description
SearchResponse<ProductWithNestedFlattened> descriptionResult = loggingOpenSearchClient.search(s -> s
.index(testIndex.getName())
.query(q -> q
.term(t -> t
.field("details.description")
.value(FieldValue.of("Premium quality product"))
)
),
ProductWithNestedFlattened.class
);
assertThat(descriptionResult.hits().total().value()).isEqualTo(1);
assertThat(descriptionResult.hits().hits().get(0).source().getId()).isEqualTo("1");
// Search combining nested and direct properties: details.attribute.color="red" AND details.description="Compact design"
SearchResponse<ProductWithNestedFlattened> combinedResult = loggingOpenSearchClient.search(s -> s
.index(testIndex.getName())
.query(q -> q
.bool(b -> b
.must(m -> m
.term(t -> t
.field("details.attribute.color")
.value(FieldValue.of("red"))
)
)
.must(m -> m
.term(t -> t
.field("details.description")
.value(FieldValue.of("Compact design"))
)
)
)
),
ProductWithNestedFlattened.class
);
assertThat(combinedResult.hits().total().value()).isEqualTo(1);
assertThat(combinedResult.hits().hits().get(0).source().getId()).isEqualTo("3");
}
}
See what’s happening here? Flattened fields support nested structures and multi-level dotted notation beautifully. You can search on:
- Nested properties:
details.attribute.color,details.attribute.size - Direct properties:
details.description - Combined queries: matching both nested and direct properties together
The flattened field type preserves the entire object hierarchy, allowing you to access properties at any nesting level using the appropriate number of dots in your query path. It’s like having a map to navigate through your data structure!
HTTP Commands for OpenSearch Dashboards
Here’s the index creation request:
PUT /flat_object_nested
{
"mappings": {
"properties": {
"details": {
"type": "flat_object"
}
}
},
"settings": {
"number_of_replicas": 0,
"number_of_shards": 1
}
}
Index Documents Request:
POST /flat_object_nested/_bulk
{ "index" : { "_index" : "flat_object_nested", "_id" : "1" } }
{ "id" : "1", "name" : "Product1", "details" : { "attribute" : { "color" : "red", "size" : "large" }, "description" : "Premium quality product" } }
{ "index" : { "_index" : "flat_object_nested", "_id" : "2" } }
{ "id" : "2", "name" : "Product2", "details" : { "attribute" : { "color" : "blue", "size" : "medium" }, "description" : "Standard quality product" } }
{ "index" : { "_index" : "flat_object_nested", "_id" : "3" } }
{ "id" : "3", "name" : "Product3", "details" : { "attribute" : { "color" : "red", "size" : "small" }, "description" : "Compact design" } }
Search Request 1: Nested property search (details.attribute.color=“red”)
POST flat_object_nested/_search
{
"query": {
"term": {
"details.attribute.color": {
"value": "red"
}
}
}
}
Search Response 1:
{
"took": 10,
"timed_out": false,
"_shards": {
"failed": 0,
"skipped": 0,
"successful": 1,
"total": 1
},
"hits": {
"hits": [
{
"_id": "1",
"_index": "flat_object_nested",
"_score": 0.646255,
"_source": {
"id": "1",
"name": "Product1",
"details": {
"attribute": {
"color": "red",
"size": "large"
},
"description": "Premium quality product"
}
}
},
{
"_id": "3",
"_index": "flat_object_nested",
"_score": 0.646255,
"_source": {
"id": "3",
"name": "Product3",
"details": {
"attribute": {
"color": "red",
"size": "small"
},
"description": "Compact design"
}
}
}
],
"max_score": 0.6462550163269043,
"total": {
"relation": "eq",
"value": 2
}
}
}
Search Request 2: Another nested property (details.attribute.size=“large”)
POST flat_object_nested/_search
{
"query": {
"term": {
"details.attribute.size": {
"value": "large"
}
}
}
}
Search Response 2:
{
"took": 2,
"timed_out": false,
"_shards": {
"failed": 0,
"skipped": 0,
"successful": 1,
"total": 1
},
"hits": {
"hits": [
{
"_id": "1",
"_index": "flat_object_exact_match",
"_score": 1.3486402,
"_source": {
"id": "1",
"name": "Product1",
"details": {
"attribute": {
"color": "red",
"size": "large"
},
"description": "Premium quality product"
}
}
}
],
"max_score": 1.3486402034759521,
"total": {
"relation": "eq",
"value": 1
}
}
}
Search Request 3: Direct property search (details.description)
POST flat_object_nested/_search
{
"query": {
"term": {
"details.description": {
"value": "Premium quality product"
}
}
}
}
Search Response 3:
{
"took": 4,
"timed_out": false,
"_shards": {
"failed": 0,
"skipped": 0,
"successful": 1,
"total": 1
},
"hits": {
"hits": [
{
"_id": "1",
"_index": "flat_object_nested",
"_score": 1.3486402,
"_source": {
"id": "1",
"name": "Product1",
"details": {
"attribute": {
"color": "red",
"size": "large"
},
"description": "Premium quality product"
}
}
}
],
"max_score": 1.3486402034759521,
"total": {
"relation": "eq",
"value": 1
}
}
}
Search Request 4: Combined nested and direct properties
POST flat_object_nested/_search
{
"query": {
"bool": {
"must": [
{
"term": {
"details.attribute.color": {
"value": "red"
}
}
},
{
"term": {
"details.description": {
"value": "Compact design"
}
}
}
]
}
}
}
Search Response 4:
{
"took": 3,
"timed_out": false,
"_shards": {
"failed": 0,
"skipped": 0,
"successful": 1,
"total": 1
},
"hits": {
"hits": [
{
"_id": "3",
"_index": "flat_object_nested",
"_score": 1.9948952,
"_source": {
"id": "3",
"name": "Product3",
"details": {
"attribute": {
"color": "red",
"size": "small"
},
"description": "Compact design"
}
}
}
],
"max_score": 1.9948952198028564,
"total": {
"relation": "eq",
"value": 1
}
}
}
Flattened Arrays: Understanding the Limitations
Alright, now we need to talk about an important limitation. When you have an array of objects mapped as a flattened field, you cannot reliably match multiple values from the same object in that array. However, flattened arrays can match multiple values from different objects in the array. This behavior demonstrates why flattened fields don’t preserve object boundaries—they treat the array as a flat collection. This is where flattened fields differ significantly from nested fields, and we’ll explain why.
Let’s start by looking at the data structures we’ll be working with. The ProductWithFlattenedArray document class contains an array of attributes:
public class ProductWithFlattenedArray implements IDocumentWithId {
private String id;
private String name;
private List<ProductAttribute> attributes; // Array of strongly-typed records
// ... constructor and getters/setters
}
The ProductAttribute record structure used in the array is straightforward:
public record ProductAttribute(
String color,
String size
) {
}
When you index a document with this structure, OpenSearch doesn’t store it as nested JSON objects. Instead, flattened fields store all values from the array as a flat collection using dot notation, like:
["color.red", "size.large", "color.blue", "size.small"]
This flat storage means that flattened arrays lose object boundaries—values from different objects in the array are treated equally, which is why matching works across different objects but not reliably within the same object.
Now, here’s where the behavior becomes apparent. Let’s break down the test that demonstrates how flattened arrays handle matching values from different objects from FlattenedSearchingTests.java:
@Test
public void flattenedMapping_ArrayOfObjects_CannotMatchMultipleValuesFromSameObject() throws Exception {
// Create a test index with flattened mapping for the attributes array field
try (OpenSearchTestIndex testIndex = fixture.createTestIndex(mapping ->
mapping.properties("attributes", Property.of(p -> p.flatObject(f -> f))))) {
// Create a document with an array of attributes where each object has multiple properties
// Using strongly-typed ProductAttribute records
List<ProductAttribute> attributes = new ArrayList<>();
attributes.add(new ProductAttribute("red", "large")); // Object 1: red and large
attributes.add(new ProductAttribute("blue", "small")); // Object 2: blue and small
ProductWithFlattenedArray[] products = new ProductWithFlattenedArray[]{
new ProductWithFlattenedArray("1", "Product1", attributes)
};
testIndex.indexDocuments(products);
// Search for color="red" AND size="small" from DIFFERENT objects
// This WILL match because flattened fields treat arrays as flat collections
// - attributes.color="red" matches (from first object)
// - attributes.size="small" matches (from second object)
// Both values exist in the array, so the document matches even though they're from different objects
SearchResponse<ProductWithFlattenedArray> differentObjectsResult = loggingOpenSearchClient.search(s -> s
.index(testIndex.getName())
.query(q -> q
.bool(b -> b
.must(m -> m
.term(t -> t
.field("attributes.color")
.value(FieldValue.of("red"))
)
)
.must(m -> m
.term(t -> t
.field("attributes.size")
.value(FieldValue.of("small"))
)
)
)
),
ProductWithFlattenedArray.class
);
// This SHOULD match because both "red" and "small" exist in the array
// (even though they're from different objects)
assertThat(differentObjectsResult.hits().total().value()).isEqualTo(1);
assertThat(differentObjectsResult.hits().hits().get(0).source().getId()).isEqualTo("1");
// Verify that individual searches also work
SearchResponse<ProductWithFlattenedArray> colorResult = loggingOpenSearchClient.search(s -> s
.index(testIndex.getName())
.query(q -> q
.term(t -> t
.field("attributes.color")
.value(FieldValue.of("red"))
)
),
ProductWithFlattenedArray.class
);
assertThat(colorResult.hits().total().value()).isEqualTo(1);
// This demonstrates that flattened arrays don't preserve object boundaries:
// - Matching values from DIFFERENT objects works (as shown above)
// - But you cannot reliably ensure values come from the SAME object
// For accurate matching of multiple properties from the same array object,
// you must use nested or join field types
}
}
Here’s the key insight: when you search for attributes.color="red" AND attributes.size="small", the query will match because both values exist in the array—even though “red” comes from the first object and “small” comes from the second object. This demonstrates that flattened fields don’t preserve object boundaries within arrays.
The problem is: flattened arrays cannot reliably match multiple values from a single object in the array because they can’t distinguish between values coming from the same object versus different objects. If you need that kind of precision (ensuring both “red” and “large” come from the exact same array object), you’ll need nested fields (which we’ll cover in future posts).
HTTP Commands for OpenSearch Dashboards
Here’s the index creation request:
PUT /flat_object_array
{
"mappings": {
"properties": {
"attributes": {
"type": "flat_object"
}
}
},
"settings": {
"number_of_replicas": 0,
"number_of_shards": 1
}
}
Index Documents Request:
POST /flat_object_array/_bulk
{ "index" : { "_index" : "flat_object_array", "_id" : "1" } }
{ "id" : "1", "name" : "Product1", "attributes" : [ { "color" : "red", "size" : "large" }, { "color" : "blue", "size" : "small" } ] }
Search Request 1: Matching values from different objects (color=“red” AND size=“small”)
POST flat_object_array/_search
{
"query": {
"bool": {
"must": [
{
"term": {
"attributes.color": {
"value": "red"
}
}
},
{
"term": {
"attributes.size": {
"value": "small"
}
}
}
]
}
}
}
Search Response 1:
{
"took": 14,
"timed_out": false,
"_shards": {
"failed": 0,
"skipped": 0,
"successful": 1,
"total": 1
},
"hits": {
"hits": [
{
"_id": "1",
"_index": "flat_object_array",
"_score": 0.83003354,
"_source": {
"id": "1",
"name": "Product1",
"attributes": [
{
"color": "red",
"size": "large"
},
{
"color": "blue",
"size": "small"
}
]
}
}
],
"max_score": 0.830033540725708,
"total": {
"relation": "eq",
"value": 1
}
}
}
Search Request 2: Individual search (attributes.color=“red”)
POST flat_object_array/_search
{
"query": {
"term": {
"attributes.color": {
"value": "red"
}
}
}
}
Search Response 2:
{
"took": 2,
"timed_out": false,
"_shards": {
"failed": 0,
"skipped": 0,
"successful": 1,
"total": 1
},
"hits": {
"hits": [
{
"_id": "1",
"_index": "flat_object_array",
"_score": 0.41501677,
"_source": {
"id": "1",
"name": "Product1",
"attributes": [
{
"color": "red",
"size": "large"
},
{
"color": "blue",
"size": "small"
}
]
}
}
],
"max_score": 0.415016770362854,
"total": {
"relation": "eq",
"value": 1
}
}
}
Flattened vs. Nested: When to Use Each
Understanding when to use flattened fields versus nested fields is crucial for building effective search solutions. Don’t worry—we’ll help you figure out which one fits your needs. Let’s compare them:
Flattened Fields (OS flat object, ES flattened) – Best For:
- Single objects with multiple properties you want to search on
- Multiple independent flattened fields in the same document
- Mapping explosion prevention when you have dynamic object structures
- Simple queries on object properties using dotted notation
- Keyword-based searching (since flattened fields index everything as keywords)
Nested Fields – Best For:
- Arrays of objects where you need to match multiple properties from the same object
- Complex queries that require maintaining object relationships within arrays
- Queries like “find products where an attribute has color=‘red’ AND size=‘large’ in the same object”
- Cases where object boundaries matter for query accuracy
The fundamental difference is that nested fields preserve object boundaries, allowing you to query within the context of a single nested object, while flattened fields lose this context for arrays.
Sloth Summary
Phew! We’ve covered a lot of ground in this article. Let’s recap what we’ve learned about the flattened field type (OS flat object, ES flattened) in OpenSearch:
- What it is: Flattened fields treat entire objects as single fields, indexing all sub-properties as keywords
- Why use it: Prevents mapping explosion, reduces memory overhead, and simplifies index management for dynamic object structures
- Simple usage: Create flattened mappings with
Property.of(p -> p.flatObject(f -> f))and search using dotted notation - Multiple fields: You can have multiple flattened fields in the same document and search across them independently
- Critical limitation: Flattened arrays cannot reliably match multiple values from the same object in the array—for that, you need nested fields
The flattened field type (OS flat object, ES flattened) is a powerful tool in your OpenSearch arsenal, but like any tool, it’s important to understand its strengths and limitations. Use flattened fields for single objects or multiple independent flattened fields, but reach for nested fields when you need to maintain object relationships within arrays. Remember: a wise Code Sloth chooses the right tool for the job! 🦥
You can find all the code examples we discussed in the Code Sloth Code Samples page. The FlattenedDemo package contains all the tests and document classes we explored today.
Happy Code Slothing! 🦥
