SoylentNews

MrPlow writes:

Submitted via IRC for SoyCow1984

Facebook AI Research is open-sourcing some of the conversational AI tech it is using to power its Portal video chat display and M suggestions on Facebook Messenger.

The company announced today that its PyTorch-based PyText NLP framework is now available to developers.

Natural language processing deals with how systems parse human language and are able to make decisions and derive insights. The PyText framework, which the company sees as a conduit for AI researchers to move more quickly between experimentation and deployment, will be particularly useful for tasks like document classification, sequence tagging, semantic parsing and multitask modeling, among others, Facebook says.

Source: https://techcrunch.com/2018/12/14/facebook-open-sources-pytext-natural-language-processing-framework/

Original Submission

takyon writes:

Mozilla updates Common Voice dataset with 1,400 hours of speech across 18 languages

Mozilla wants to make it easier for startups, researchers, and hobbyists to build voice-enabled apps, services, and devices. Toward that end, it's today releasing the latest version of Common Voice, its open source collection of transcribed voice data that now comprises over 1,400 hours of voice samples from 42,000 contributors across 18 languages, including English, French, German, Dutch, Hakha-Chin, Esperanto, Farsi, Basque, Spanish, Mandarin Chinese, Welsh, and Kabyle.

It's one of the largest multi-language dataset of its kind, Mozilla claims — substantially larger than the Common Voice corpus it made publicly available eight months ago, which contained 500 hours (400,000 recordings) from 20,000 volunteers in English — and the corpus will soon grow larger still. The organization says that data collection efforts in 70 languages are actively underway via the Common Voice website and mobile apps.

Common Voice home page. Also at Engadget.

Previously: Mozilla's "Common Voice": Voice Recognition Without Google, Amazon, Baidu, Apple, Microsoft, etc.
Mozilla's Common Voice Collecting French, German, and Welsh Samples, Prepping 40 More Languages

Original Submission

upstart writes:

Submitted via IRC for SoyCow2718

Investigating the self-attention mechanism behind BERT-based architectures

BERT, a transformer-based model characterized by a unique self-attention mechanism, has so far proved to be a valid alternative to recurrent neural networks (RNNs) in tackling natural language processing (NLP) tasks. Despite their advantages, so far, very few researchers have studied these BERT-based architectures in depth, or tried to understand the reasons behind the effectiveness of their self-attention mechanism.

Aware of this gap in the literature, researchers at the University of Massachusetts Lowell's Text Machine Lab for Natural Language Processing have recently carried out a study investigating the interpretation of self-attention, the most vital component of BERT models. The lead investigator and senior author for this study were Olga Kovaleva and Anna Rumshisky, respectively. Their paper pre-published on arXiv and set to be presented at the EMNLP 2019 conference, suggests that a limited amount of attention patterns are repeated across different BERT sub-components, hinting to their over-parameterization.

"BERT is a recent model that made a breakthrough in the NLP community, taking over the leaderboards across multiple tasks. Inspired by this recent trend, we were curious to investigate how and why it works," the team of researchers told TechXplore via email. "We hoped to find a correlation between self-attention, the BERT's main underlying mechanism, and linguistically interpretable relations within the given input text."