|
JAIST Repository >
f. 情報社会基盤研究センター >
f10. 学術雑誌論文等 >
f10-1. 雑誌掲載論文 >
このアイテムの引用には次の識別子を使用してください:
http://hdl.handle.net/10119/3305
|
| タイトル: | Maximum entropy models with inequality constraints: A case study on text categorization |
| 著者: | Kazama, J
Tsujii, J |
| キーワード: | maximum entropy model
inequality constraint
regularization
feature selection
text categorization |
| 発行日: | 2005-09 |
| 出版者: | Springer Science + Business Media |
| 誌名: | Machine Learning |
| 巻: | 60 |
| 号: | 1-3 |
| 開始ページ: | 159 |
| 終了ページ: | 194 |
| 抄録: | Data sparseness or overfitting is a serious problem in natural language processing employing machine learning methods. This is still true even for the maximum entropy (ME) method, whose flexible modeling capability has alleviated data sparseness more successfully than the other probabilistic models in many NLP tasks. Although we usually estimate the model so that it completely satisfies the equality constraints on feature expectations with the ME method, complete satisfaction leads to undesirable overfitting, especially for sparse features, since the constraints derived from a limited amount of training data are always uncertain. To control overfitting in ME estimation, we propose the use of box-type inequality constraints, where equality can be violated up to certain predefined levels that reflect this uncertainty. The derived models, inequality ME models, in effect have regularized estimation with L_1 norm penalties of bounded parameters. Most importantly, this regularized estimation enables the model parameters to become sparse. This can be thought of as automatic feature selection, which is expected to improve generalization performance further. We evaluate the inequality ME models on text categorization datasets, and demonstrate their advantages over standard ME estimation, similarly motivated Gaussian MAP estimation of ME models, and support vector machines (SVMs), which are one of the state-of-the-art methods for text categorization. |
| Rights: | This is the author-created version of Springer Netherlands, Jun'ichi Kazama and Jun'ichi Tsujii, Machine Learning, 60(1-3), 2005, 159-194. The original publication is available at www.springerlink.com, http://www.springerlink.com/content/v6mn41027420jp56/fulltext.pdf |
| URI: | http://hdl.handle.net/10119/3305 |
| 資料タイプ: | author |
| 出現コレクション: | f10-1. 雑誌掲載論文 (Journal Articles)
|
このアイテムのファイル:
| ファイル |
記述 |
サイズ | 形式 |
|---|
| S-64.pdf | | 409Kb | Adobe PDF | 見る/開く |
|
当システムに保管されているアイテムはすべて著作権により保護されています。
|
