commit 2169b0bc896d048a567eeb9e5cbb53927a62dde6
author Noriyuki Takahashi <noriyukit@google.com> Sat Jan 31 09:08:12 2015 +0000
committer Yohei Yukawa <yukawa@google.com> Sat Jan 31 09:08:12 2015 +0000
tree 78f1323dd7a614cb0011d08717ef64ca116f9e49
parent 25b997d798dbe08d2b2dbd9414ed3747e1918889

Refactor SystemDictionary::PrefixSearch using LoudsTrie::Node APIs

This CL re-implements prefix search without LoudsTrie::Callback, which simplifies code because the nested callback of DictionaryInterface::Callback and LoudsTrie::Callback was removed.

Also, some performance improvements are implemented.
- Instead of decoding the prefix using Codec::DecodeKey, only its length is decoded by Codec::GetDecodeKeyLength and the decoded prefix is obtained from the original user input.  This is faster because we can avoid UCS4 to UTF8 conversion and string construction.
- Lazy decoding of LoudsTrie's key ID.  In the previous implementation, key IDs were always decoded due to the restriction of LoudsTrie::Callback.  However, key ID is first used when starting decoding token array elements.  Thus, when performing culling based on key strings, we can skip decoding of key ID.
- Optimization for the case where key expansion is not performed (e.g., predictor calls LookupPrefix without expansion).  In the previous implementation, this case was implemented by passing the default KeyExpansionTable.  However, since recursive traversal over LoudsTrie is far complicated than the normal traversal, it's worth optimizing the code for non-expanding cases.

Roughly 10% time reduction is observed in converter_benchmark with several corpora.  Note that this is just a code refactoring, no user-visible behavior change is intended.

BUG=none
TEST=unittest,benchmark

git-svn-id: https://mozc.googlecode.com/svn/trunk@525 a6090854-d499-a067-5803-1114d4e51264