Why did we remove multiple document types within an index in ElasticSearch?

Question

The answer is this-: Because we can't declare a field of different data types within a same index in different document types.  Say there's an index called 'college'.  Then there are document types called 'student' 'teacher' 'administration' 'staff'.  What problem would occur if we allow this?    Books and documentations are saying that if a field called 'date_of_join' is given a 'text' data type in 'student', then we can't give 'date_of_join' as 'date' data type in 'staff'.   It says that it's due to the way Lucene is.   This is because of the way Lucene maintains the field types in an index. As Lucene manages fields on an index level, there is no flexibility to declare two fields of different data types in the same index   But this is not clear without an example(of how lucene is storing index). Can you guys clarify this?   I know that lucene stores inverted indexes though. But still I'm not clear.

vikas_jk · Accepted Answer

Once you define field in Lucene Elastic Search, it is considered of same datatype internally in there system, so basically if was issue in there system, that you cannot have 2 columns of different tables with different datatype but same name, so they decided to remove this limitations from ES 8

Check this:

In an SQL database, tables are independent of each other. The columns in one table have no bearing on columns with the same name in another table. This is not the case for fields in a mapping type.

In an Elasticsearch index, fields that have the same name in different mapping types are backed by the same Lucene field internally. In other words, using the example above, the user_name field in the user type is stored in exactly the same field as the user_name field in the tweet type, and both user_name fields must have the same mapping (definition) in both types.

This can lead to frustration when, for example, you want deleted to be a date field in one type and a boolean field in another type in the same index.

On top of that, storing different entities that have few or no fields in common in the same index leads to sparse data and interferes with Lucene’s ability to compress documents efficiently.

For these reasons, we have decided to remove the concept of mapping types from Elasticsearch.

Try looking at these articles

https://www.elastic.co/guide/en/elasticsearch/reference/7.17/removal-of-types.html

https://stackoverflow.com/questions/35725618/should-create-multiple-document-types-or-multiple-indexes

.you need to check that Elastic search have removed Index from version 8

Why did we remove multiple document types within an index in ElasticSearch?

Subscribe Now