Why did we remove multiple document types within an index in ElasticSearch?


The answer is this-: Because we can't declare a field of different data types within a same index in different document types.

Say there's an index called "college".

Then there are document types called "student" "teacher" "administration" "staff".

What problem would occur if we allow this? 

Books and documentations are saying that if a field called "date_of_join" is given a "text" data type in "student", then we can't give "date_of_join" as "date" data type in "staff".

It says that it's due to the way Lucene is.

This is because of the way Lucene maintains the field types in an index. As Lucene manages fields on an index level, there is no flexibility to declare two fields of different data types in the same index

But this is not clear without an example(of how lucene is storing index). Can you guys clarify this? 

I know that lucene stores inverted indexes though. But still I'm not clear.


Asked by:- shivajikobardan
0
: 127 At:- 7/19/2022 9:11:27 AM
lucene elasticsearch







1 Answers
profileImage Answered by:- vikas_jk

Once you define field in Lucene Elastic Search, it is considered of same datatype internally in there system, so basically if was issue in there system, that you cannot have 2 columns of different tables with different datatype but same name, so they decided to remove this limitations from ES 8

Check this:

In an SQL database, tables are independent of each other. The columns in one table have no bearing on columns with the same name in another table. This is not the case for fields in a mapping type.

In an Elasticsearch index, fields that have the same name in different mapping types are backed by the same Lucene field internally. In other words, using the example above, the user_name field in the user type is stored in exactly the same field as the user_name field in the tweet type, and both user_name fields must have the same mapping (definition) in both types.

This can lead to frustration when, for example, you want deleted to be a date field in one type and a boolean field in another type in the same index.

On top of that, storing different entities that have few or no fields in common in the same index leads to sparse data and interferes with Lucene’s ability to compress documents efficiently.

For these reasons, we have decided to remove the concept of mapping types from Elasticsearch.

Try looking at these articles

https://www.elastic.co/guide/en/elasticsearch/reference/7.17/removal-of-types.html

https://stackoverflow.com/questions/35725618/should-create-multiple-document-types-or-multiple-indexes

.you need to check that Elastic search have removed Index from version 8

1
At:- 7/19/2022 11:34:44 AM
I kind of need 1 example of what problem occurs in lucene to be clear about this issue. 0
By : shivajikobardan - at :- 7/19/2022 12:19:25 PM
Google " Elastic search mapping", but you will get mapping examples not the issue example, like here https://logz.io/blog/elasticsearch-mapping/ 0
By : vikas_jk - at :- 7/19/2022 12:26:10 PM
yup i've looked, there's only high level information about it. Say there's an index called "college". Then there are document types called "student" "teacher" "administration" "staff". What problem would occur if we allow this? 0
By : shivajikobardan - at :- 7/23/2022 1:50:20 AM
You can simply understand like this it will not work, will give you runtime error, not compile time. 0
By : vikas_jk - at :- 7/23/2022 6:18:11 AM






Login/Register to answer
Or
Register directly by posting answer/details

Full Name *

Email *




By posting your answer you agree on privacy policy & terms of use

Subscribe Now

Subscribe to our weekly Newsletter & Keep getting latest article/questions in your inbox weekly