TextRazor's API helps you rapidly build state-of-the-art language processing technology into your application.
Our main analysis endpoint offers a simple combined call that allows you to perform several different analyses on the same document, for example extracting both the entities mentioned in the text and relations between them. The API allows callers to specify a number of extractors, which control the range of TextRazor's language analysis features.
If you have any queries please contact us at support@textrazor.com and we will get back to you promptly. We'd also love to hear from you if you have any ideas for improving the API or documentation.
The TextRazor API identifies each of your requests by your unique API Key, which you can find in the console.
All endpoints expect the API Key to be passed as an HTTP header - "X-TextRazor-Key: YOUR_API_KEY".
When calling the public TextRazor API you should always use the secure HTTPS endpoint to protect your credentials.
To enable compression of the TextRazor response JSON, add the Accept-encoding: 'gzip' header to your request. This can significantly reduce the size of TextRazor responses, and is recommended for most requests if possible.
curl -X POST \
-H "x-textrazor-key: YOUR_API_KEY" \
-d "extractors=entities,entailments" \
-d "text=Spain's stricken Bankia expects to sell off its vast portfolio of industrial holdings that includes a stake in the parent company of British Airways and Iberia." \
https://api.textrazor.com/
TextRazor will return an HTTP error when it encounters a problem processing your text. A more detailed message is provided in the response body.
200: OK
400: Bad Request
401: Unauthorized
413: Request too large
500: Internal Server Error
Please check the TextRazor status page for updates on any system-wide issues.
To handle any intermittent connection issues we recommend that you design your application to gracefully retry sending failed requests several times before stopping and logging the error. TextRazor operates a resilient high-availability infrastructure, so any potential disruption should be kept to a minimum.
TextRazor was designed to work out of the box with a wide range of different types of content. However there are several steps you can take to help improve the system further for your specific application:
Please do not hesitate to contact us for help with getting the most out of the system for your use case.
Main endpoint for all analysis functions.
TextRazor expects all analysis POST requests to be "application/x-www-form-urlencoded" encoding, as used by forms on the web. Requests should include at least "url" or "text" paramaters, as well as one or more "extractors" which control the operations that TextRazor performs on your document.
Requests should also always include the "X-TextRazor-Key: YOUR_API_KEY" HTTP header to identify your account.
text
url
extractors
Sets a list of “Extractors”, which tells TextRazor which analysis functions to perform on your text. For optimal performance, only select the extractors that are explicitly required by your application.
rules
cleanupHTML
cleanup.mode
Controls the preprocessing cleanup mode that TextRazor will apply to your content before analysis. For all options aside from "raw" any position offsets returned will apply to the final cleaned text, not the raw HTML. If the cleaned text is required please see the cleanup_return_cleaned option.
cleanup.returnCleaned
cleanup.returnRaw
cleanup.useMetadata
When use_metadata is True, TextRazor will use metadata extracted from your document to help in the disambiguation/extraction process. This include HTML titles and metadata, and can significantly improve results for shorter documents without much other content.
This option has no effect when cleanup_mode is 'raw'. Defaults to True.
download.userAgent
Sets the User-Agent header to be used when downloading over HTTP. This should be a descriptive string identifying your application, or an end user's browser user agent if you are performing live requests from a given user.
Defaults to "TextRazor Downloader (https://www.textrazor.com)"
languageOverride
entities.dictionaries
Sets a list of the custom entity dictionaries to match against your content. Each item should be a string ID corresponding to dictionaries you have previously configured through the DictionaryManager interface.
entities.filterDbpediaTypes
entities.filterFreebaseTypes
entities.allowOverlap
classifiers
Sets a list of classifiers to evaluate against your document. Each entry should be a string ID corresponding to either one of TextRazor's default classifiers, or one you have previously configured through the ClassifierManager interface.
If you aren't tied to a particular taxonomy version, the current textrazor_mediatopics_2023Q1 is a sound starting point for many classification projects.
time
response
ok
error
message
customAnnotationOutput
cleanedText
rawText
entailments
Entailment across all sentences in the response.
entities
Entity across all sentences in the response.
topics
Topic in the response.
categories
ScoredCategory in the response.
nounPhrases
NounPhrase in the response.
properties
Property across all sentences in the response.
relations
Relation across all sentences in the response.
sentences
Sentence in the response.
matchingRules
language
languageIsReliable
{
"time":"TIME",
"response":{
"cleanedText":"CLEANED_TEXT",
"sentences":[SENTENCE_JSON, ...],
"entities":[ENTITY_JSON, ...],
"categories":[CATEGORY_JSON, ...],
"topics":[TOPIC_JSON, ...],
"entailments":[ENTAILMENT_JSON, ...],
"relations":[RELATION_JSON, ...],
"properties":[PROPERTY_JSON, ...],
"nounPhrases":[NOUNPHRASE_JSON, ...],
}
}
entityId
entityEnglishId
customEntityId
confidenceScore
type
freebaseTypes
freebaseId
wikidataId
matchingTokens
matchedText
data
relevanceScore
wikiLink
Represents a single “Named Entity” extracted from text.
Each entity is disambiguated to Wikipedia and Freebase concepts wherever possible. Where the entity could not be linked the relevant properties will return None.
Entities are returned with both Confidence and Relevance scores when possible. These measure slightly different things. The confidence score is a measure of the engine's confidence that the entity is a valid entity given the document context, whereas the relevance score measures how on-topic or important that entity is to the document. As an example, a news story mentioning "Barack Obama" in passing would assign high confidence to the "President" entity. If the story isn't about politics, however, the same entity might have a low relevance score.
Scores can vary if the same entity is mentioned more than once. As an entity is mentioned in different contexts the engine will report different scores.
label
score
wikiLink
wikidataId
Represents a single “Topic” extracted from text.
categoryId
label
score
The score TextRazor has assigned to this category, between 0 and 1.
To avoid false positives you might want to ignore categories below a certain score - a good starting point would be 0.5. The best way to find an appropriate threshold is to run a sample set of your documents through the system and manually inspect the results.
classifierId
Represents a single “Category” that matches your document.
contextScore
entailedTree
wordPositions
priorScore
score
Represents a single “entailment” derived from the source text.
Please note - If you need the source word for each Entailment you must request
the "words" extractor.
wordPositions
relation
wordPositions
Represents a multi-word phrase extracted from a sentence.
To extract the full text of the noun phrase from the original content you must add the "words" extractor, and use the word offsets to recreate the original string.
wordPositions
propertyPositions
Represents a property relation extracted from raw text. A property implies an “is-a” or “has-a” relationship between the predicate (or focus) and its property.
params
RelationParam of this relation.
wordPositions
Represents a grammatical relation between words. Typically owns a number of RelationParam, representing the SUBJECT and OBJECT of the relation.
To extract the full text of the relation predicate or param from the original content you must add the "words" extractor, and use the word offsets to recreate the original string.
endingPos
startingPos
lemma
parentPosition
partOfSpeech
senses
spellingSuggestions
position
relationToParent
stem
token
Represents a single Word (token) extracted by TextRazor.
words
Word in this sentence.
Represents a single sentence extracted by TextRazor.
TextRazor Entity Dictionaries allow you to augment the TextRazor entity extraction system with custom entities that are relevant to your application.
Entity Dictionaries are useful for identifying domain specific entities that may not be common enough for TextRazor to know about out of the box - examples might be Product names, Drug names, and specific person names.
TextRazor supports flexible, high performance matching of dictionaries up to several million entries, limited only by your account plan. Entries are automatically indexed and distributed across our analysis infrastructure to ensure they scale with your application.
Once you have created a dictionary, add its ID to your analysis requests with the 'entityDictionaries' option. TextRazor will look for any DictionaryEntry in the dictionary that can be matched to your document, and return it as part of the standard Entity response.
All match text, and other properties, are expected to be valid UTF-8. Dictionary Entries can match any UTF-8 character.
Creates a new dictionary using properties provided in the JSON body.
See the properties of class Dictionary for valid options.
curl -XPUT -H 'x-textrazor-key: YOUR_API_KEY' \
-d '{"matchType":"token", "caseInsensitive":true, "language":"eng"}' \
https://api.textrazor.com/entities/test_ents
Returns a list of all Dictionary in your account.
curl -H 'x-textrazor-key: YOUR_API_KEY' \
https://api.textrazor.com/entities/
Returns a Dictionary object by id.
curl -H 'x-textrazor-key: YOUR_API_KEY' \
https://api.textrazor.com/entities/test_ents
Deletes a dictionary and all its entries by id.
curl -XDELETE -H 'x-textrazor-key: YOUR_API_KEY' \
https://api.textrazor.com/entities/test_ents
Returns all entities in this dictionary, along with paging information.
Larger dictionaries can be too large to download all at once. Where possible it is recommended that you use limit and offset paramaters to control the TextRazor response, rather than filtering client side.
curl -H 'x-textrazor-key: YOUR_API_KEY' \
https://api.textrazor.com/entities/test_ents/_all?limit=20&offset=0
Adds entries to this dictionary.
Entries must be a JSON array of objects corresponding to properties of the new DictionaryEntry objects. At a minimum this would be [{'text':'test text to match'}].
Each entry assigned to a Dictionary is uniquely identified by the 'id' property. Where this is not provided a unique id is automatically generated by the server. When an entity is added with an id that matches an existing entity, the old entity is overwritten. This behaviour allows you to keep your entity dictionaries up-to-date without any downtime from deleting and recreating dictionaries.
curl -XPOST -H 'x-textrazor-key: YOUR_API_KEY' \
-d '[{"text":"Bjarne Stroustrup","id":"DEV2"}]' https://api.textrazor.com/entities/test_ents/
Gets a specific entity by id.
curl -H 'x-textrazor-key: YOUR_API_KEY' \
https://api.textrazor.com/entities/test_ents/DEV2
Deletes a specific entity by id.
For performance reasons it's always better to delete and recreate a whole dictionary rather than its individual entries one at a time.
curl -XDELETE -H 'x-textrazor-key: YOUR_API_KEY' \
https://api.textrazor.com/entities/test_ents/DEV2
Users on any of our paid plans can create up to 10 dictionaries, with a total of 10,000 entries. TextRazor supports custom dictionaries of millions of entries, please contact us to discuss increasing this limit for your account.
Free account holders are able to create 1 Dictionary with a total of 50 Entries.
matchType
Controls any pre-processing done on your dictionary before matching.
Defaults to ''.
caseInsensitive
When True, this dictionary will match both uppercase and lowercase characters.
Defaults to ''
id
The unique identifier for this dictionary.
language
When set to a ISO-639-2 language code, this dictionary will only match documents of the corresponding language.
When set to 'any', this dictionary will match any document.
Defaults to ''
Represents a single Dictionary, uniqely identified by an id. Each Dictionary owns a set of DictionaryEntry.
Dictionary and DictionaryEntry can only be manipulated through the DictionaryManager object.
Represents a single dictionary entry, belonging to a Dictionary object.
id
Unique ID for this entry, used to identify and manipulate specific entries.
Defaults to an automatically generated unique id.
text
String representing the text to match to this DictionaryEntry.
data
A dictionary mapping string keys to lists of string data values. Where TextRazor matches this entry to your content in analysis, it will return the dictionary as part of the entity response.
This is useful for adding application-specific metadata to each entry. Dictionary data is limited to a maximum of 10 keys, and a total of 1000 characters across all the mapped values.
{'type':['people', 'person', 'politician']}
TextRazor can classify your documents according to the IPTC Newscode or IAB QAG taxonomies using our predefined models.
Sometimes the categories you might be interested in aren't well represented by off-the-shelf classifiers. TextRazor gives you the flexibility to create a customized model for your particular project.
TextRazor uses "concept queries" to define new categories. These are similar to the sort of boolean query that you might type into a search engine, except they query the semantic meaning of the document you are analyzing. Each concept query uses a word or two in English to define your category.
For an example of how to create a custom classifier please see our tutorials. If you aren't getting the results you need, please contact us, we'd be happy to help.
The ClassifierManager interface offers a simple interface for creating and managing your classifiers. Classifiers only need to be uploaded once, they are safetly stored on our servers to use with future analyze requests. Simply add the classifier name to your request's "classifiers" list.
If an application/json Content-Type header is passed with the request, TextRazor creates a new classifier using categories provided in the JSON body. See the properties of class Category for valid options.
If a classifier with this ID already exists, the categories are added to the existing classifier. New categories will replace any categories with the same ID.
curl -XPUT -H 'x-textrazor-key: YOUR_API_KEY' \
-H 'Content-Type: application/json' \
-d '[category_json,..]'' \
https://api.textrazor.com/categories/sport
If an application/csv Content-Type header is passed with the request, TextRazor expects your request body to contain a CSV file. This should be formatted with each row corresponding to a category, and each column representing a category field.
The new classifier will replace any existing classifier with the same name.
The format should be as follows:
Category ID,Category Label,Category Query
classifer_file.csv:
100,Golf,concept('sport>golf')
101,Squash,concept('sport>squash')
102,Cricket,concept('sport>cricket')
curl -XPUT -H 'Content-Type: application/csv' \
-H 'x-textrazor-key: YOUR_API_KEY' \
--data-binary '@classifier_file.csv' \
https://api.textrazor.com/categories/sport
Deletes a Classifier and all its Categories by id.
curl -X DELETE \
-H 'x-textrazor-key: YOUR_API_KEY' \
https://api.textrazor.com/categories/sport
Returns a list of all Categories for classifier with id classifier_id, along with paging information.
Larger classifiers can be too large to download all at once. Where possible it is recommended that you use limit and offset paramaters to control the TextRazor response, rather than filtering client side.
curl -H 'x-textrazor-key: YOUR_API_KEY' \
https://api.textrazor.com/categories/sport/_all?limit=20&offset=0
Returns a Category object by id.
curl -H 'x-textrazor-key: YOUR_API_KEY' \
https://api.textrazor.com/categories/sport/101
Deletes a Category object by id.
For performance reasons it's always better to delete and recreate a whole classifier rather than its individual categories one at a time.
curl -X DELETE \
-H 'x-textrazor-key: YOUR_API_KEY' \
https://api.textrazor.com/categories/sport/101
Users on any of our paid plans can create up to 10 Classifiers, with a total of 1000 categories. Please contact us to discuss increasing this limit for your account.
Free account holders are able to create 1 Classifier with a total of 50 Categories.
There are no restrictions on the use of classifiers that have been pre-defined by TextRazor.
categoryId
label
query
Represents a single Category that belongs to a Classifier. Each category consists of a unique ID, and a query at a minimum.
{
"categoryId" : "100",
"label" : "Golf",
"query" : "concept('sport>golf')"
}
AccountManager allows you to retrieve data about your TextRazor account, designed to help manage and control your usage.
Returns a complete Account object.
curl -X GET \
-H 'x-textrazor-key: YOUR_API_KEY' \
https://api.textrazor.com/account/
plan
concurrentRequestLimit
concurrentRequestsUsed
planDailyIncludedRequests
requestsUsedToday