apache / carbondata / 422

76%

push

jenkins

[CARBONDATA-2417] [SDK Writer] SDK writer goes to infinite wait when consumer thread goes dead

Problem:
SDK Writer going into infinte loop in case of multi-threaded scenario

Analysis:
In multi-threaded scenarios where multiple instances of writer thread are trying to add a row to the RowBatch, addition to given batch size cannot be ensured as addition process is not synchronized and it can lead to ArrayIndexOutOfBound Exception or data loss/mismatch issues.
If multiple writer threads are adding the data to RowBatch and immediately after launching all the threads closeWriter is called, in that case we don't know when all the data is finished writing by all the threads but we are returning immediately from close writer after setting the close flag to true. This does not ensure complete processing of data.

Solution:
Make the row addition logic synchronized and modify the code in closeWriter to ensure data completeness.

This closes #2286

1 of 1 new or added line in 1 file covered. (100.0%)

51496 of 68799 relevant lines covered (74.85%)

1.0 hits per line