TextRazor's API helps you rapidly build state-of-the-art language processing technology into your application.
Our main analysis endpoint offers a simple combined call that allows you to perform several different analyses on the same document, for example extracting both the entities mentioned in the text and relations between them. The API allows callers to specify a number of extractors, which control the range of TextRazor's language analysis features.
If you have any queries please contact us at support@textrazor.com and we will get back to you promptly. We'd also love to hear from you if you have any ideas for improving the API or documentation.
The easiest way to install the TextRazor Java SDK is with Maven. To add the dependency to your project simply add the following to your pom.xml:
<dependency> <groupId>com.textrazor</groupId> <artifactId>textrazor</artifactId> <version>1.0.12</version> </dependency>
Alternatively you can pick up the latest source or precompiled Jar from GitHub.
TextRazor for Java depends on the Jackson JSON library. Ensure that this is on your classpath when running your project (using Maven as above takes care of this for you).
import com.textrazor.TextRazor;
import com.textrazor.annotations.Entity;
import com.textrazor.annotations.AnalyzedText;
TextRazor client = new TextRazor(API_KEY);
client.addExtractor("words");
client.addExtractor("entities");
AnalyzedText response = client.analyze("LONDON - Barclays misled shareholders and the public RBS about one of the biggest investments in the bank's history, a BBC Panorama investigation has found.");
for (Entity entity : response.getResponse().getEntities()) {
System.out.println("Matched Entity: " + entity.getEntityId());
}
Please see https://github.com/TextRazor/textrazor-java/blob/master/test/com/textrazor/TestTextRazor.java for a more detailed example of the TextRazor Java SDK.
The TextRazor API identifies each of your requests by your unique API Key, which you can find in the console.
By default the Java SDK uses SSL connections to encrypt all communication with the TextRazor server.
The TextRazor Java SDK throws an AnalysisException whenever it is unable to process your request, or a NetworkException
whenever it was unable to connect to TextRazor.
These both come with a descriptive message explaining the problem.
We recommend that you design your application to gracefully retry sending failed requests several times before stopping and logging the error.
TextRazor was designed to work out of the box with a wide range of different types of content. However there are several steps you can take to help improve the system further for your specific application:
Please do not hesitate to contact us for help with getting the most out of the system for your use case.
All TextRazor functionality is exposed in the TextRazor class. Integrating into your project is simple. Create a TextRazor instance with your
API key and the extractors you are interested in, then call analyze for each of your documents.
This class is threadsafe once initialized with the request options. You should create a new instance for each request if you are likely to be changing the request options in a multithreaded environment.
AnalyzedText analyze(java.lang.String text)
Calls the TextRazor API with the provided UTF8 encoded text. Returns a Response with the analyzed metadata on success. Raises an AnalysisException or NetworkException on failure.
AnalyzedText analyze(java.lang.String text)
Calls the TextRazor API with the provided url.
TextRazor will first download the contents of this URL, and then process the resulting text.
TextRazor will only attempt to analyze text documents. Any invalid UTF-8 characters will be replaced with a space character and ignored. TextRazor limits the total download size to approximately 1M. Any larger documents will be truncated to that size, and a warning will be returned in the response.
By default, TextRazor will clean all HTML prior to processing. For more control of the cleanup process, see the setCleanupMode option.
Returns a Response with the analyzed metadata on success. Raises an AnalysisException or NetworkException on failure.
setExtractors(java.util.List<java.lang.String> extractors)
Sets a list of “Extractors”, which tells TextRazor which analysis functions to perform on your text. For optimal performance, only select the extractors that are explicitly required by your application.
setRules(java.lang.String rules)
setCleanupHTML(boolean cleanupHTML)
setCleanupMode(java.lang.String cleanupMode)
Controls the preprocessing cleanup mode that TextRazor will apply to your content before analysis. For all options aside from "raw" any position offsets returned will apply to the final cleaned text, not the raw HTML. If the cleaned text is required please see the cleanup_return_cleaned option.
setCleanupReturnCleaned(boolean cleanupReturnCleaned)
setCleanupReturnRaw(boolean cleanupReturnRaw)
setCleanupUseMetadata(boolean cleanupUseMetadata)
When use_metadata is True, TextRazor will use metadata extracted from your document to help in the disambiguation/extraction process. This include HTML titles and metadata, and can significantly improve results for shorter documents without much other content.
This option has no effect when cleanup_mode is 'raw'. Defaults to True.
setDownloadUserAgent(java.lang.String downloadUserAgent)
Sets the User-Agent header to be used when downloading over HTTP. This should be a descriptive string identifying your application, or an end user's browser user agent if you are performing live requests from a given user.
Defaults to "TextRazor Downloader (https://www.textrazor.com)"
setLanguageOverride(java.lang.String languageOverride)
setDoCompression(boolean doCompression)
setDoEncryption(boolean doEncryption)
setEntityDictionaries(List<String> entityDictionaryIds)
Sets a list of the custom entity dictionaries to match against your content. Each item should be a string ID corresponding to dictionaries you have previously configured through the DictionaryManager interface.
setDbpediaTypeFilters(java.util.List<java.lang.String> dbpediaTypeFilters)
setFreebaseTypeFilters(java.util.List<java.lang.String> freebaseTypeFilters)
setAllowOverlap(boolean allowOverlap)
setClassifiers(List<String> classifierIds)
Sets a list of classifiers to evaluate against your document. Each entry should be a string ID corresponding to either one of TextRazor's default classifiers, or one you have previously configured through the ClassifierManager interface.
If you aren't tied to a particular taxonomy version, the current textrazor_mediatopics_2023Q1 is a sound starting point for many classification projects.
getTime()
isOk()
getError()
getMessage()
getCustomAnnotationOutput()
getEntailments()
Entailment across all sentences in the response.
getEntities()
Entity across all sentences in the response.
getTopics()
Topic in the response.
getCategories()
ScoredCategory in the response.
getNounPhrases()
NounPhrase in the response.
getProperties()
Property across all sentences in the response.
getRelations()
Relation across all sentences in the response.
getSentences()
Sentence in the response.
getLanguage()
getLanguageIsReliable()
All calls to analyze return a AnalyzedText object.
This in turn contains a Response object with the response data.
getEntityId()
getEntityEnglishId()
getCustomEntityId()
getConfidenceScore()
getDBPediaTypes()
getFreebaseTypes()
getFreebaseId()
getWikidataId()
getMatchingTokens()
getMatchingWords()
Word that make up this entity.
getMatchedText()
getData()
getRelevanceScore()
getWikiLink()
Represents a single “Named Entity” extracted from text.
Each entity is disambiguated to Wikipedia and Freebase concepts wherever possible. Where the entity could not be linked the relevant properties will return None.
Entities are returned with both Confidence and Relevance scores when possible. These measure slightly different things. The confidence score is a measure of the engine's confidence that the entity is a valid entity given the document context, whereas the relevance score measures how on-topic or important that entity is to the document. As an example, a news story mentioning "Barack Obama" in passing would assign high confidence to the "President" entity. If the story isn't about politics, however, the same entity might have a low relevance score.
Scores can vary if the same entity is mentioned more than once. As an entity is mentioned in different contexts the engine will report different scores.
getLabel()
getScore()
getWikiLink()
getWikidataId()
Represents a single “Topic” extracted from text.
getCategoryId()
getLabel()
getScore()
The score TextRazor has assigned to this category, between 0 and 1.
To avoid false positives you might want to ignore categories below a certain score - a good starting point would be 0.5. The best way to find an appropriate threshold is to run a sample set of your documents through the system and manually inspect the results.
getClassifierId()
Represents a single “Category” that matches your document.
getContextScore()
getEntailedWords()
getEntailedTree()
getWordPositions()
getMatchedWords()
Word in the current sentence that generated this entailment.
getPriorScore()
getScore()
Represents a single “entailment” derived from the source text.
Please note - If you need the source word for each Entailment you must request
the "words" extractor.
getWordPositions()
getParamWords()
Word that make up this param.
getRelation()
getWordPositions()
getWords()
Word that make up this phrase.
Represents a multi-word phrase extracted from a sentence.
To extract the full text of the noun phrase from the original content you must add the "words" extractor, and use the word offsets to recreate the original string.
getWordPositions()
getPropertyWords()
getPropertyPositions()
getPropertyWords()
Word that make up the property that targets the focus words.
Represents a property relation extracted from raw text. A property implies an “is-a” or “has-a” relationship between the predicate (or focus) and its property.
getParams()
RelationParam of this relation.
getWordPositions()
getPredicateWords()
Word in this relation.
Represents a grammatical relation between words. Typically owns a number of RelationParam, representing the SUBJECT and OBJECT of the relation.
To extract the full text of the relation predicate or param from the original content you must add the "words" extractor, and use the word offsets to recreate the original string.
getChildren()
Word that make up the children of this word. Returns an empty list for leaf words, or if the “dependency-trees” extractor was not requested.
getEntailments()
Entailment that this word entails.
getEntities()
Entity that this word is a part of.
getEndingPos()
getStartingPos()
getLemma()
getNounPhrases()
NounPhrase that this word is a member of.
getParentWord()
getParentPosition()
getPartOfSpeech()
getSenses()
getSpellingSuggestions()
getPosition()
getPropertyPredicates()
Property that this word is a predicate (or focus) member of.
getRelationParams()
RelationParam that this word is a member of.
getRelationToParent()
getRelations()
Relation that this word is a predicate of.
getStem()
getToken()
Represents a single Word (token) extracted by TextRazor.
For convenience the Java SDK automatically creates helper functions to retrieve annotations extracted from that sentence.
getWords()
Word in this sentence.
Represents a single sentence extracted by TextRazor.
TextRazor Entity Dictionaries allow you to augment the TextRazor entity extraction system with custom entities that are relevant to your application.
Entity Dictionaries are useful for identifying domain specific entities that may not be common enough for TextRazor to know about out of the box - examples might be Product names, Drug names, and specific person names.
TextRazor supports flexible, high performance matching of dictionaries up to several million entries, limited only by your account plan. Entries are automatically indexed and distributed across our analysis infrastructure to ensure they scale seamlessly with your application.
Once you have created a dictionary, add its ID to your analysis requests with setEntityDictionaries
. TextRazor will look for any DictionaryEntry in the dictionary that
can be matched to your document, and return it as part of the standard Entity response.
createDictionary(Dictionary dictionary)
Creates a new dictionary.
See the properties of class Dictionary for valid options.
import com.textrazor.dictionary.DictionaryManager;
DictionaryManager manager = new DictionaryManager(apiKey);
manager.createDictionary(Dictionary.builder().setId('developers').setMatchType('token').build());
allDictionaries()
Returns a list of all Dictionary in your account.
for (Dictionary dict : manager.allDictionaries()) {
System.out.println('Current dictionary: ' + dict.getId());
}
getDictionary(String id)
Returns a Dictionary object by id.
Dictionary dict = manager.getDictionary('developers');
deleteDictionary(String id)
Deletes a dictionary and all its entries by id.
manager.deleteDictionary('developers');
allEntries(String id, int limit, int offset)
Returns a AllDictionaryEntriesResponse containing all DictionaryEntry for a dictionary, along with paging information.
Larger dictionaries can be too large to download all at once. Where possible it is recommended that you use limit and offset paramaters to control the TextRazor response, rather than filtering client side.
PagedAllEntries allEntries = manager.allEntries('developers');
addEntries(String id, List<DictionaryEntry> entries)
Adds entries to a dictionary.
Entries must be a list corresponding to properties of the new DictionaryEntry objects. At a minimum this would be [{'text':'test text to match'}].
ListnewEntries = new ArrayList (); List types = Arrays.asList('cpp_developer', 'writer'); newEntries.add(DictionaryEntry.builder().setText('Bjarne Stroustrup').setId('DEV2').addData('types', types).build()); manager.addEntries(newDict.getId(), newEntries);
getEntry(String dictionaryId, String entryId)
Retrieves a specific DictionaryEntry by dictionary id and entry id.
DictionaryEntry entry = manager.getEntry('developers', 'DEV2');
deleteEntry(String dictionaryId, String entryId)
Deletes a specific DictionaryEntry by dictionary id and entry id.
For performance reasons it's always faster to perform major changes to dictionaries by deleting and recreating the whole dictionary rather than removing many individual entries.
manager.deleteEntry('developers', 'DEV2');
Users on any of our paid plans can create up to 10 dictionaries, with a total of 10,000 entries. TextRazor supports custom dictionaries of millions of entries, please contact us to discuss increasing this limit for your account.
Free account holders are able to create 1 Dictionary with a total of 50 Entries.
getMatchType()
Controls any pre-processing done on your dictionary before matching.
Defaults to ''.
getCaseInsensitive()
When True, this dictionary will match both uppercase and lowercase characters.
Defaults to ''
getId()
The unique identifier for this dictionary.
getLanguage()
When set to a ISO-639-2 language code, this dictionary will only match documents of the corresponding language.
When set to 'any', this dictionary will match any document.
Defaults to ''
Represents a single Dictionary, uniqely identified by an id. Each Dictionary owns a set of DictionaryEntry.
Dictionary and DictionaryEntry can only be manipulated through the DictionaryManager object.
Represents a single dictionary entry, belonging to a Dictionary object.
getId()
Unique ID for this entry, used to identify and manipulate specific entries.
Defaults to an automatically generated unique id.
getText()
String representing the text to match to this DictionaryEntry.
getData()
A dictionary mapping string keys to lists of string data values. Where TextRazor matches this entry to your content in analysis, it will return the dictionary as part of the entity response.
This is useful for adding application-specific metadata to each entry. Dictionary data is limited to a maximum of 10 keys, and a total of 1000 characters across all the mapped values.
{'type':['people', 'person', 'politician']}
TextRazor can classify your documents according to the IPTC Media Topics, IPTC Newscode or IAB QAG taxonomies using our predefined models.
Sometimes the categories you might be interested in aren't well represented by off-the-shelf classifiers. TextRazor gives you the flexibility to create a customized model for your particular project.
TextRazor uses "concept queries" to define new categories. These are similar to the sort of boolean query that you might type into a search engine, except they query the semantic meaning of the document you are analyzing. Each concept query uses a word or two in English to define your category.
For an example of how to create a custom classifier please see our tutorials. If you aren't getting the results you need, please contact us, we'd be happy to help.
The ClassifierManager interface offers a simple interface for creating and managing your classifiers. Classifiers only need to be uploaded once, they are safetly stored on our servers to use with future analyze requests. Simply add the classifier name to your request's "classifiers" list.
createClassifier(String classifierId, List<Category> categories)
Creates a new classifier using the provided list of Category.
See the properties of class Category for valid options.
import com.textrazor.classifier.ClassifierManager;
ClassifierManager manager = new ClassifierManager(apiKey);
String classifierId = "my_test_classifier";
manager.createClassifier(testClassifierId, Arrays.asList(Category.builder().setCategoryId("Soccer").setQuery("or(concept('soccer'),concept('association football'))").build()));
deleteClassifier(String classifierId)
Deletes a Classifier and all its Categories by id.
manager.deleteClassifier(testClassifierId);
allCategories(String classifierId, int limit, int offset)
Returns a AllCategoriesResponse containing all Category for a classifier, along with paging information.
Larger classifiers can be too large to download all at once. Where possible it is recommended that you use limit and offset paramaters to control the TextRazor response, rather than filtering client side.
for (Category cat : manager.allCategories(testClassifierId).getCategories()) {
System.out.println(cat.getCategoryId() + " " + cat.getQuery() + " " + cat.getLabel());
}
deleteCategory(String classifierId, String categoryId)
Deletes a Category object by id.
For performance reasons it's always better to delete and recreate a whole classifier rather than its individual categories one at a time.
manager.deleteCategory(testClassifierId, "Soccer")
getCategory(String classifierId, String categoryId)
Returns a Category object by id.
System.out.println(manager.getCategory(testClassifierId, "Soccer").getLabel());
Users on any of our paid plans can create up to 10 Classifiers, with a total of 1000 categories. Please contact us to discuss increasing this limit for your account.
Free account holders are able to create 1 Classifier with a total of 50 Categories.
There are no restrictions on the use of classifiers that have been pre-defined by TextRazor.
getCategoryId()
getLabel()
getQuery()
Represents a single Category that belongs to a Classifier. Each category consists of a unique ID, and a query at a minimum.
{
"categoryId" : "100",
"label" : "Golf",
"query" : "concept('sport>golf')"
}
Allows you to retrieve data about your TextRazor account, designed to help manage and control your usage.
getAccount()
Returns a complete Account object.
The account endpoint is read only. Calls to this endpoint do not count towards your daily quota.
getPlan()
getConcurrentRequestLimit()
getConcurrentRequestsUsed()
getPlanDailyIncludedRequests()
getRequestsUsedToday()