XAI / Language Models - Versionsgeschichte

Mattis: huggingface

2020-02-14T07:14:05Z

huggingface

← Nächstältere Version		Version vom 14. Februar 2020, 09:14 Uhr
Zeile 170:		Zeile 170:
	https://huggingface.co/		https://huggingface.co/
	Writer: https://transformer.huggingface.co/doc/distil-gpt2		Writer: https://transformer.huggingface.co/doc/distil-gpt2


	Diskussion (old):		Diskussion (old):
Zeile 177:		Zeile 178:
	* https://blog.floydhub.com/gpt2/		* https://blog.floydhub.com/gpt2/
	----		----

	======'''BERT'''======		======'''BERT'''======
	bidirectional encoder representations from Transformers https://arxiv.org/abs/1810.04805		bidirectional encoder representations from Transformers https://arxiv.org/abs/1810.04805

Mattis: /* Newer techniques */

2020-02-14T07:13:21Z

Newer techniques

← Nächstältere Version		Version vom 14. Februar 2020, 09:13 Uhr
Zeile 166:		Zeile 166:
	** Improving Language Understanding by Generative Pre-Training https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf		** Improving Language Understanding by Generative Pre-Training https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf
	** https://github.com/openai/gpt-2		** https://github.com/openai/gpt-2

			======'''Weiterentwicklung / Implementierung'''======
			https://huggingface.co/
			Writer: https://transformer.huggingface.co/doc/distil-gpt2

	Diskussion (old):		Diskussion (old):

C.heck: /* (un-)supervised techniques */

2020-01-14T21:56:14Z

(un-)supervised techniques

← Nächstältere Version		Version vom 14. Januar 2020, 23:56 Uhr
Zeile 149:		Zeile 149:

	* »on the road« by AI: https://medium.com/artists-and-machine-intelligence/ai-poetry-hits-the-road-eb685dfc1544		* »on the road« by AI: https://medium.com/artists-and-machine-intelligence/ai-poetry-hits-the-road-eb685dfc1544
			----
	====Autoencoder====		====Autoencoder====
	https://www.wired.co.uk/article/google-artificial-intelligence-poetry		https://www.wired.co.uk/article/google-artificial-intelligence-poetry
			----
	====LSTM+Autoencoder====		====LSTM+Autoencoder====
	* https://github.com/keras-team/keras/issues/1401		* https://github.com/keras-team/keras/issues/1401
	* https://www.dlology.com/blog/how-to-do-unsupervised-clustering-with-keras/		* https://www.dlology.com/blog/how-to-do-unsupervised-clustering-with-keras/
			----
	====GAN====		====GAN====
	https://arxiv.org/abs/1705.10929		https://arxiv.org/abs/1705.10929
			----
	====Newer techniques====		====Newer techniques====
	====='''transformer-based language models'''=====		====='''transformer-based language models'''=====
Zeile 172:		Zeile 172:
	How to Build...(old):		How to Build...(old):
	* https://blog.floydhub.com/gpt2/		* https://blog.floydhub.com/gpt2/
			----
	======'''BERT'''======		======'''BERT'''======
	bidirectional encoder representations from Transformers https://arxiv.org/abs/1810.04805		bidirectional encoder representations from Transformers https://arxiv.org/abs/1810.04805

			----
	====='''LSTM-based language models'''=====		====='''LSTM-based language models'''=====
	======'''ELMO'''======		======'''ELMO'''======
	Deep contextualized word representations https://arxiv.org/abs/1802.05365 < uses bidirectional LSTM		Deep contextualized word representations https://arxiv.org/abs/1802.05365 < uses bidirectional LSTM
			----
	======'''GRU''' (Gated Recurrent Unit)======		======'''GRU''' (Gated Recurrent Unit)======
	it uses so-called, update gate and reset gate https://arxiv.org/abs/1406.1078		it uses so-called, update gate and reset gate https://arxiv.org/abs/1406.1078

C.heck: /* chatbots */

2020-01-14T21:51:15Z

chatbots

← Nächstältere Version		Version vom 14. Januar 2020, 23:51 Uhr
Zeile 202:		Zeile 202:

	===chatbots===		===chatbots===
	~~<big>~~siehe [[Chatbots\|Chatbot-exMediaWiki-Seite]]~~</big>~~		'''siehe [[Chatbots\|Chatbot-exMediaWiki-Seite]]'''

	----		----

C.heck: /* datenbanken */

2020-01-14T21:50:54Z

datenbanken

← Nächstältere Version		Version vom 14. Januar 2020, 23:50 Uhr
Zeile 187:		Zeile 187:

	===datenbanken===		===datenbanken===
			muß upgedated werden!!

	deutsch:		deutsch:
	* https://de.wikipedia.org/wiki/GermaNet		* https://de.wikipedia.org/wiki/GermaNet
Zeile 198:		Zeile 200:

	----		----

	===chatbots===		===chatbots===
	<big>siehe [[Chatbots\|Chatbot-exMediaWiki-Seite]]</big>		<big>siehe [[Chatbots\|Chatbot-exMediaWiki-Seite]]</big>

C.heck: /* chatbots */

2020-01-14T21:48:36Z

chatbots

← Nächstältere Version		Version vom 14. Januar 2020, 23:48 Uhr
Zeile 199:		Zeile 199:
	----		----
	===chatbots===		===chatbots===
	* https://bdtechtalks.com/2017/08/21/rob-high-ibm-watson-cto-artificial-intelligence-chatbots/		<big>siehe [[Chatbots\|Chatbot-exMediaWiki-Seite]]</big>
	* https://chatbotsmagazine.com/contextual-chat-bots-with-tensorflow-4391749d0077
	** Jupyter Notebook: https://github.com/ugik/notebooks/blob/master/Tensorflow%20chat-bot%20model.ipynb
	* Facebook-~~Messenger~~-~~Bot: https://dzone.com/articles/how-i-used-deep-learning-to-train-a-chatbot-to-tal~~
	** Git-Repo: https://github.com/adeshpande3/Facebook-Messenger-Bot
	* https://tutorials.botsfloor.com/how-to-build-your-first-chatbot-c84495d4622d
	** Jupyter Notebooks: https://github.com/suriyadeepan/~~practical_seq2seq~~

	----		----

	===Toolkits/Librarys===		===Toolkits/Librarys===
	* Natural Language Toolkit: http://www.nltk.org/		* Natural Language Toolkit: http://www.nltk.org/

C.heck: /* Newer techniques */

2020-01-14T21:46:22Z

Newer techniques

← Nächstältere Version		Version vom 14. Januar 2020, 23:46 Uhr
Zeile 161:		Zeile 161:

	====Newer techniques====		====Newer techniques====
	=====transformer-based language models=====		====='''transformer-based language models'''=====
	======'''OpenAI's gpt-2'''======		======'''OpenAI's gpt-2'''======
	* https://openai.com/blog/better-language-models/		* https://openai.com/blog/better-language-models/
Zeile 177:		Zeile 177:


	=====LSTM-based language models=====		====='''LSTM-based language models'''=====
	======'''ELMO'''======		======'''ELMO'''======
	Deep contextualized word representations https://arxiv.org/abs/1802.05365 < uses bidirectional LSTM		Deep contextualized word representations https://arxiv.org/abs/1802.05365 < uses bidirectional LSTM

C.heck: /* Newer techniques */

2020-01-14T21:45:45Z

Newer techniques

← Nächstältere Version		Version vom 14. Januar 2020, 23:45 Uhr
Zeile 173:		Zeile 173:
	* https://blog.floydhub.com/gpt2/		* https://blog.floydhub.com/gpt2/

	======BERT======		======'''BERT'''======
	bidirectional encoder representations from Transformers https://arxiv.org/abs/1810.04805		bidirectional encoder representations from Transformers https://arxiv.org/abs/1810.04805


	=====LSTM-based language models=====		=====LSTM-based language models=====
	======ELMO======		======'''ELMO'''======
	Deep contextualized word representations https://arxiv.org/abs/1802.05365 < uses bidirectional LSTM		Deep contextualized word representations https://arxiv.org/abs/1802.05365 < uses bidirectional LSTM

	======GRU (Gated Recurrent Unit)======		======'''GRU''' (Gated Recurrent Unit)======
	it uses so-called, update gate and reset gate https://arxiv.org/abs/1406.1078		it uses so-called, update gate and reset gate https://arxiv.org/abs/1406.1078
	* https://towardsdatascience.com/understanding-gru-networks-2ef37df6c9be		* https://towardsdatascience.com/understanding-gru-networks-2ef37df6c9be

C.heck: /* Newer techniques */

2020-01-14T21:45:08Z

Newer techniques

← Nächstältere Version		Version vom 14. Januar 2020, 23:45 Uhr
Zeile 162:		Zeile 162:
	====Newer techniques====		====Newer techniques====
	=====transformer-based language models=====		=====transformer-based language models=====
	======OpenAI's gpt-2======		======'''OpenAI's gpt-2'''======
	* https://openai.com/blog/better-language-models/		* https://openai.com/blog/better-language-models/
	** Improving Language Understanding by Generative Pre-Training https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf		** Improving Language Understanding by Generative Pre-Training https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf

C.heck: /* (un-)supervised techniques */

2020-01-14T21:44:33Z

(un-)supervised techniques

← Nächstältere Version		Version vom 14. Januar 2020, 23:44 Uhr
Zeile 160:		Zeile 160:
	https://arxiv.org/abs/1705.10929		https://arxiv.org/abs/1705.10929

	====transformer-based language ~~model~~====		====Newer techniques====
	OpenAI's gpt-2:		=====transformer-based language models=====
			======OpenAI's gpt-2======
	* https://openai.com/blog/better-language-models/		* https://openai.com/blog/better-language-models/
			** Improving Language Understanding by Generative Pre-Training https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf
	** https://github.com/openai/gpt-2		** https://github.com/openai/gpt-2

	Diskussion:		Diskussion (old):
	* https://www.skynettoday.com/briefs/gpt2		* https://www.skynettoday.com/briefs/gpt2

	How to Build...:		How to Build...(old):
	* https://blog.floydhub.com/gpt2/		* https://blog.floydhub.com/gpt2/

			======BERT======
			bidirectional encoder representations from Transformers https://arxiv.org/abs/1810.04805


			=====LSTM-based language models=====
			======ELMO======
			Deep contextualized word representations https://arxiv.org/abs/1802.05365 < uses bidirectional LSTM

			======GRU (Gated Recurrent Unit)======
			it uses so-called, update gate and reset gate https://arxiv.org/abs/1406.1078
			* https://towardsdatascience.com/understanding-gru-networks-2ef37df6c9be
	----		----