Today, I try the Mecab to deal with Japanese word segmentation problem.
But the buffer-errors comes out.
input-buffer overflow. The line is split. use -b #SIZE option.
we can set a bigger buffer to fixed this problem.
for example, the default size is 8192, we can change it to 8192 * 5
mecab -b 40960 < data
