What is natural language processing?
Natural language text processing—general direction
Analysis applied to artificial intelligencemeans understanding the language, and synthesis means generating literate text. Solving these problems will mean creating a more convenient form of interaction between a computer and a person.
Objectives and Limitations
Theoretically, the construction of a natural languageinterface for computers is a very attractive target. Early systems such as SHRDLU, working with a limited “cube world” and using a limited vocabulary, looked extremely good, inspiring their creators. However, optimism quickly dwindled when these systems were confronted with the complexity and ambiguity of the real world.
Natural language comprehension is sometimes consideredAI is a complete task, because recognition of a living language requires a huge knowledge of the system about the surrounding world and the ability to interact with it. The very definition of the meaning of the word "understand" is one of the main tasks of artificial intelligence.
Difficulties in understanding the Russian language
The quality of understanding depends on many factors: from the language, from the national culture, from the interlocutor himself, etc. These are some examples of the difficulties that text understanding systems face.
- Difficulties with revealing anaphors (recognition,what is meant by the use of pronouns): the sentences “We gave the bananas to the monkeys because they were hungry” and “We gave the bananas to the monkeys because they were overripe” are similar in syntactic structure. In one of them the pronounthey arerefers to monkeys, and in another to bananas. Correct understanding depends on the computer's knowledge of what bananas and monkeys can be.
- The free order of words can lead to a completely different interpretation of the phrase: "Being determines consciousness" - what determines what?
- In Russian, free order is compensated by a developed morphology, official words and punctuation marks, but in most cases this presents an additional problem for a computer.
- Neologisms can be encountered in speech, for example, the verb "Fifty ruble" - that is, send 50 rubles. The system should be able to distinguish such cases from typos and understand them correctly.
- Correct understanding of homonyms is another problem. In speech recognition, among others, the problem of phonetic homonyms arises. In the phrase “The gray wolf in the wildernessforestmet a redheadfox»Highlighted words are heard the same way, and withoutknowledge of who is deaf and who is red is indispensable (except that the fox can be red and the forest can be deaf, the forest can also be red (a characteristic, in this case, denoting the predominant color of the foliage in the forest), while the fox may be deaf, which gives rise to an additional problem arising from the previous one, although it is partially compensated by morphology - the adjectives in this sentence are clearly different in gender).
Popular tasks:
- Speech recognition
- Text analysis:
- Extraction of information,
- Information search,
- Analysis of statements,
- Sentiment analysis of the text,
- Question-answer systems.
- Generating text
- Speech synthesis
General classification:
- Categorization of texts
- Classification of character sequences:
- Named Entity Recognition,
- Determination of parts of speech of words.
- Phrase Recognition
- Extracting information from text
- Syntactic annotation
- Semantic annotation
- Generating text:
- Generation of text based on recognized speech,
- Machine translate,
- Generalization of the text.
How does writing AI use it at work?
- The washington post
In August 2016, The Washington Post for the first timebegan to use the bot Heliograf, which wrote short news about the Olympic Games in Rio de Janeiro. The performance of "Heliograf" was impressive: the bot generated news faster than the editor had time to set the task, and the readers could not distinguish automatic notes from handwritten ones.
- Bloomberg
About 30% of all Bloomberg news todayare created using the Cyborg module. It generates them according to the template: what happened, when, where, with whom, who and how commented on the event. This saves costs for reporters, but you can't do without them. Cyborg is just an automation system, not an advanced AI.
- Reuters
International agency Reuters uses softwareNews Tracer. It is an AI predictive tool that evaluates Twitter stories based on statistical and reputation criteria. The bot checks over 700 million tweets every day.
- The guardian
For the first time, artificial intelligence has gone beyondnews notes and began to generate analytical articles. In January 2019, The Guardian published the first story written by the artificial intelligence ReporterMate. It was dedicated to the amount of donations collected by various parties in Australia. In addition to the text, the AI generated graphs and rated the games based on the results of the training camp
Read more:
The first accurate map of the world was created. What's wrong with everyone else?
The most stormy place on Earth: why Drake Passage is the most dangerous route to Antarctica
New uranium compound breaks record for anomalous conductivity