Coupled Pattern Learner (CPL) is a machine learning algorithm which couples the semi-supervised learning of categories and relations to forestall the problem of semantic drift associated with boot-strap learning methods.

Coupled Pattern Learner

Semi-supervised learning approaches using a small number of labeled examples with many unlabeled examples are usually unreliable as they produce an internally consistent, but incorrect set of extractions. CPL solves this problem by simultaneously learning classifiers for many different categories and relations in the presence of an ontology defining constraints that couple the training of these classifiers. It was introduced by Andrew Carlson, Justin Betteridge, Estevam R. Hruschka Jr. and Tom M. Mitchell in 2009.^[1][2]

CPL overview

CPL is an approach to semi-supervised learning that yields more accurate results by coupling the training of many information extractors. Basic idea behind CPL is that semi-supervised training of a single type of extractor such as ‘coach’ is much more difficult than simultaneously training many extractors that cover a variety of inter-related entity and relation types. Using prior knowledge about the relationships between these different entities and relations CPL makes unlabeled data as a useful constraint during training. For e.g., ‘coach(x)’ implies ‘person(x)’ and ‘not sport(x)’.

CPL description

Input: An ontology O, and a text corpus C 
Output: Trusted instances/patterns for each predicate
for i=1,2,...,∞ do
    foreach predicate p in O do
        EXTRACT candidate instances/contextual patterns using recently promoted patterns/instances;
        FILTER candidates that violate coupling;
        RANK candidate instances/patterns;
        PROMOTE top candidates;
    end
end

Meta-Bootstrap Learner

Meta-Bootstrap Learner (MBL) was also proposed by the authors of CPL.^[2] Meta-Bootstrap learner couples the training of multiple extraction techniques with a multi-view constraint, which requires the extractors to agree. It makes addition of coupling constraints on top of existing extraction algorithms, while treating them as black boxes, feasible. MBL assumes that the errors made by different extraction techniques are independent. Following is a quick summary of MBL.

Input: An ontology O, a set of extractors ε
Output: Trusted instances for each predicate
for i=1,2,...,∞ do
    foreach predicate p in O do
        foreach extractor e in ε do
            Extract new candidates for p using e with recently promoted instances;
        end
        FILTER candidates that violate mutual-exclusion or type-checking constraints;
        PROMOTE candidates that were extracted by all extractors;
    end
end

Subordinate algorithms used with MBL do not promote any instance on their own, they report the evidence about each candidate to MBL and MBL is responsible for promoting instances.

Applications

In their paper ^[1] authors have presented results showing the potential of CPL to contribute new facts to existing repository of semantic knowledge, Freebase ^[3]

See also

• Co-training
• Never-Ending Language Learning

Notes

References

• Liu, Qiuhua; Xuejun Liao; Lawrence Carin (2008). "Semi-supervised multitask learning". NIPS.
• Shinyama, Yusuke; Satoshi Sekine (2006). "Preemptive information extraction using unrestricted relation discovery". HLT-Naacl.
• Chang, Ming-Wei; Lev-Arie Ratinov; Dan Roth (2007). "Guiding semi-supervision with constraint driven learning". ACL.
• Banko, Michele; Michael J. Cafarella; Stephen Soderland; Matt Broadhead; Oren Etzioni (2007). "Open information extraction from the web". IJCAI.
• Blum, Avrim; Tom Mitchell (1998). "Combining labeled and unlabeled data with co-training". Proceedings of the eleventh annual conference on Computational learning theory. pp. 92–100. doi:10.1145/279943.279962. ISBN 1581130570. S2CID 207228399. {{cite book}}: |journal= ignored (help)
• Riloff, Ellen; Rosie Jones (1999). "Learning dictionaries for information extraction by multi-level bootstrapping". AAAI.
• Rosenfeld, Benjamin; Ronen Feldman (2007). "Using corpus statistics on entities to improve semi-supervised relation extraction from the web". ACL.
• Wang, Richard C.; William W. Cohen (2008). "Iterative set expansion of named entities using the web". ICDM.