Jump to content

User:Stellaathena/sandbox1

From Wikipedia, the free encyclopedia

EleutherAI
Type of businessResearch Co-operative
Founded3 July 2020; 4 years ago (2020-07-03)[1]
Founder(s)
  • Connor Leahy
  • Leo Gao
  • Sid Black
Key people
  • Stella Biderman
  • Aran Komatsuzaki
  • Ben Wang
IndustryArtificial intelligence
ProductsGPT-Neo, GPT-J, the Pile
URLeleuther.ai

EleutherAI is an artificial intelligence research laboratory founded in July 2020. EleutherAI made headlines in 2021 after its GPT-3 replication project produced the most powerful autoregressive language model freely available online.

History

[edit]

EleutherAI

While EleutherAI initially turned down funding offers, preferring to use Google's TRC program to source their compute, by early 2021 they had accepted funding from CoreWeave (a small cloud computing company) and SpellML (a cloud infrastructure company) in the form of access to powerful GPU clusters that are necessary for large scale machine learning research.

Research and technologies

[edit]

According to their website, EleutherAI is a "decentralized grassroots collective of volunteer researchers, engineers, and developers focused on AI alignment, scaling, and open source AI research".[2] While they do not sell any of their technologies as products, they publish the results of their research in academic venues, write blog posts detailing their ideas and methodologies, and provide trained models for anyone to use for free.

Massive Language Models

[edit]

EleutherAI is best known for its pioneering work on developing and publicly releasing large language models. In contrast to other leading labs such as Google and OpenAI, EleutherAI t

As of November 2021, EleutherAI

Actual Cites

[edit]

[3]

[2]

[4] - EleutherAI was created with the goal of open sourcing GPT-3 - The Pile was created because no suitable training data existed - EleutherAI uses compute from TRC - The models are freely available on HF - "GPT-Neo was able to generate a coherent, almost-believable article without missing out on the central themes"

[5] - Connor, Leo, and Sid founded EleutherAI - GPT-Neo release

[6] - The Pile was used by MSFT - The Eval Harness was used by MSFT

[7] - "We think that access to large, pretrained models will enable large swathes of research that would not have been possible while such technologies are locked away behind corporate walls. For-profit entities have explicit incentives to downplay risks and discourage security probing. We want to help the wider safety and security communities access and study these new technologies" - The EleutherAI community has now expanded its activity and is working on open-source alternatives in BioML and generative art.

[8] - The EleutherAI language models hosted on Hugging Face has been used ~300,000 times each month.

Biological ML

[edit]

Following [DeepMind]]'s breakthrough application of transformers to protein folding, EleutherAI began to expand into the space. They have a stated public goal of replicating and releasing the (unreleased) AlphaFold2 model developed by DeepMind. Enroute to that goal, EleutherAI has released several technologies and papers

See also

[edit]

References

[edit]
  1. ^ "EleutherAI One Year Retrospective". 7 July 2021.
  2. ^ a b "EleutherAI Website". EleutherAI. Retrieved 1 July 2021.
  3. ^ Luitse, Dieuwertje; Denkena, Wiebke (2021). "The great transformer: Examining the role of large language models in the political economy of AI". Big Data & Society. 8 (2): 354–359. doi:10.1177/20539517211047734. ISSN 0028-0836.
  4. ^ Iyer, Abhishek (15 May 2021). "GPT-3's free alternative GPT-Neo is something to be excited about". VentureBeat. Retrieved 12 November 2021.
  5. ^ Wiggers, Kyle (15 January 2021). "Meet the people trying to replicate and open-source OpenAI's GPT-3". VentureBeat. Retrieved 12 November 2021.
  6. ^ Alvi, Ali; Kharya, Paresh. "Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, the World's Largest and Most Powerful Generative Language Model". Microsoft Research Blog (Press release). Microsoft. Retrieved 12 November 2021.
  7. ^ Benaich, Nathan; Hogarth, Ian (2021). State of AI Report (Report).
  8. ^ Benaich (2021). "EleutherAI on HuggingFace".
[edit]


= Category:Artificial intelligence laboratories Category:Deep learning Category:Applied machine learning

Retrieved from "https://en.wikipedia.org/w/index.php?title=User:Stellaathena/sandbox1&oldid=1250240241"