Search Results for "transformers"
Sort By:
Showing 209 open source projects for "transformers"
View related business solutions-
Save Up to 91% on Cloud Compute With Spot VMsRun batch jobs at 60-91% off with Spot VMs. Long-running workloads get automatic discounts with sustained use.
-
$300 Free Credits to Build on Google CloudPut your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
-
1
Transformers
State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX
Hugging Face Transformers provides APIs and tools to easily download and train state-of-the-art pre-trained models. Using pre-trained models can reduce your compute costs, carbon footprint, and save you the time and resources required to train a model from scratch. These models support common tasks in different modalities. Text, for tasks like text classification, information extraction, question answering, summarization, translation, text generation, in over 100 languages. -
2
x-transformers
A simple but complete full-attention transformer
...Proposes adding learned memory key/values prior to attending. They were able to remove feedforwards altogether and attain a similar performance to the original transformers. I have found that keeping the feedforwards and adding the memory key/values leads to even better performance. Proposes adding learned tokens, akin to CLS tokens, named memory tokens, that is passed through the attention layers alongside the input tokens. You can also use the l2 normalized embeddings proposed as part of fixnorm. ... -
3
spacy-transformers
Use pretrained transformers like BERT, XLNet and GPT-2 in spaCy
...These techniques can be used to import knowledge from raw text into your pipeline, so that your models are able to generalize better from your annotated examples. You can convert word vectors from popular tools like FastText and Gensim, or you can load in any pre trained transformer model if you install spacy-transformers. You can also do your own language model pretraining via the spacy pre train command. You can even share your transformer or another contextual embedding model across multiple components, which can make long pipelines several times more efficient. To use transfer learning, you’ll need at least a few annotated examples for what you’re trying to predict. -
4
Transformers & LLMs cheatsheet
VIP cheatsheet for Stanford's CME 295 Transformers and Large Language
Transformers & LLMs cheatsheet is an educational repository focused on transformers, large language models, and modern generative AI architectures based on Stanford CME 295 coursework. The project compiles structured notes, mathematical explanations, diagrams, and implementation references covering transformer internals, attention mechanisms, tokenization, training pipelines, and inference strategies. -
Veeam Data Platform v13.1Try Veeam Data Platform today. Experience the unified platform that's secure by design, portable by default, and proven to recover clean, fast, and anywhere.
-
5
SageMaker Hugging Face Inference Toolkit
Library for serving Transformers models on Amazon SageMaker
SageMaker Hugging Face Inference Toolkit is an open-source library for serving Transformers models on Amazon SageMaker. This library provides default pre-processing, predict and postprocessing for certain Transformers models and tasks. It utilizes the SageMaker Inference Toolkit for starting up the model server, which is responsible for handling inference requests. For the Dockerfiles used for building SageMaker Hugging Face Containers, see AWS Deep Learning Containers. ... -
6
Feature-engine
Feature engineering package with sklearn like functionality
Feature-engine is a Python library with multiple transformers to engineer and select features for use in machine learning models. Feature-engine's transformers follow Scikit-learn's functionality with fit() and transform() methods to learn the transforming parameters from the data and then transform it. -
7
SetFit
Efficient few-shot learning with Sentence Transformers
SetFit is an efficient and prompt-free framework for few-shot fine-tuning of Sentence Transformers. It achieves high accuracy with little labeled data - for instance, with only 8 labeled examples per class on the Customer Reviews sentiment dataset, SetFit is competitive with fine-tuning RoBERTa Large on the full training set of 3k examples. -
8
Adapters
A Unified Library for Parameter-Efficient Learning
Adapters is an add-on library to HuggingFace's Transformers, integrating 10+ adapter methods into 20+ state-of-the-art Transformer models with minimal coding overhead for training and inference. Adapters provide a unified interface for efficient fine-tuning and modular transfer learning, supporting a myriad of features like full-precision or quantized training (e.g. Q-LoRA, Q-Bottleneck Adapters, or Q-PrefixTuning), adapter merging via task arithmetics or the composition of multiple adapters via composition blocks, allowing advanced research in parameter-efficient transfer learning for NLP tasks. -
9
bert4torch
An elegent pytorch implement of transformers
An elegant PyTorch implement of transformers. -
Build Securely on AWS with Proven FrameworksMoving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.
-
10
DINOv3
Reference PyTorch implementation and models for DINOv3
...DINOv3 removes the need for complex augmentations or momentum encoders, streamlining the pipeline while maintaining or improving feature quality. The model supports multiple backbone architectures, including Vision Transformers (ViT), and can handle larger image resolutions with improved stability during training. The learned embeddings generalize robustly across tasks like classification, retrieval, and segmentation without fine-tuning, showing state-of-the-art transfer performance among self-supervised models. -
11
Unlimited OCR Works
Welcome the Era of One-shot Long-horizon Parsing
...The project supports single-image parsing as well as multi-page and PDF-style parsing by converting pages into images. It provides inference paths for Hugging Face Transformers, vLLM, and SGLang, which gives users several deployment options. The repository also includes example code for batch inference over image folders or PDF inputs. Overall, it is useful for researchers and developers who need advanced OCR, long-document parsing, and model-based extraction from complex visual documents. -
12
GeoAI
GeoAI: Artificial Intelligence for Geospatial Data
GeoAI is a comprehensive open-source Python package designed to integrate artificial intelligence techniques with geospatial data analysis, enabling users to perform advanced geographic modeling and visualization tasks with ease. It provides a unified framework that combines machine learning libraries such as PyTorch and Transformers with geospatial tools, allowing users to process satellite imagery, aerial photos, and vector datasets in a streamlined workflow. The platform supports a wide range of tasks including image classification, object detection, segmentation, and change detection, making it suitable for applications in environmental monitoring, urban planning, and disaster response. ... -
13
BERTopic
Leveraging BERT and c-TF-IDF to create easily interpretable topics
...Instead, we can visualize the topics that were generated in a way very similar to LDAvis. By default, the main steps for topic modeling with BERTopic are sentence-transformers, UMAP, HDBSCAN, and c-TF-IDF run in sequence. -
14
LocalAI
The free, Open Source alternative to OpenAI, Claude and others
...LocalAI can run on consumer-grade hardware and does not necessarily require a GPU, making it accessible for local development and private deployments. It integrates with multiple backends like llama.cpp, transformers, and diffusers to support different AI workloads. With its self-hosted architecture and OpenAI-compatible API, LocalAI enables developers to build secure, local-first AI applications. -
15
AiLearning-Theory-Applying
Quickly get started with AI theory and practical applications
...The project also introduces important concepts such as probability theory, linear algebra, regression models, clustering methods, and neural network architectures. Advanced sections explore modern AI topics including transformers, BERT-based natural language processing systems, and practical competition-style machine learning workflows. -
16
nanoGPT
The simplest, fastest repository for training/finetuning models
...While simple, it can still train non-trivial models on modern GPUs and generate coherent text. The project has become widely used in tutorials, courses, and experiments for people learning how transformers work under the hood. -
17
Detoxify
Trained models & code to predict toxic comments
Detoxify is a deep learning-based tool for detecting and filtering toxic language in online conversations, leveraging Transformer models for high accuracy. -
18
SwanLab
An open-source, modern-design AI training tracking and visualization
...SwanLab supports both cloud and self-hosted deployments, allowing organizations to run the system privately or integrate it into shared development environments. The platform integrates with a wide range of machine learning frameworks including PyTorch, Transformers, Keras, and other widely used training ecosystems. -
19
DFlash
Block Diffusion for Ultra-Fast Speculative Decoding
...The project includes support for multiple draft models, example integration code, and scripts to benchmark performance, and it is structured to work with popular model serving stacks like SGLang and the Hugging Face Transformers ecosystem. -
20
Attention Residuals (AttnRes)
Drop-in replacement for standard residual connections in Transformers
Attention Residuals is a research-driven architectural innovation for transformer-based models that replaces traditional residual connections with an attention-based mechanism to improve information flow across layers. In standard transformers, residual connections simply sum outputs from previous layers, which can lead to uncontrolled growth of hidden states and dilution of early-layer information in deep networks. Attention Residuals introduces a learnable softmax attention mechanism that allows each layer to selectively retrieve and weight useful representations from earlier layers, making depth dynamically adaptive rather than uniformly aggregated. ... -
21
PydanticAI
Agent Framework / shim to use Pydantic with LLMs
...PydanticAI is a Python Agent Framework designed to make it less painful to build production-grade applications with Generative AI. Built by the team behind Pydantic (the validation layer of the OpenAI SDK, the Anthropic SDK, LangChain, LlamaIndex, AutoGPT, Transformers, CrewAI, Instructor, and many more). -
22
Kimi Linear
An expressive, efficient attention architecture
...Released Base and Instruct checkpoints contain 48 billion total parameters, activate 3 billion parameters per token, and support contexts up to one million tokens. The models were trained on 5.7 trillion tokens and can reduce KV cache requirements by up to 75 percent. Kimi Linear supports inference through Hugging Face Transformers and deployment as an OpenAI-compatible API with vLLM. -
23
AI Engineering Transition Path
Research papers and blogs to transition to AI Engineering
AI Engineering Resources is an open educational repository that compiles research papers, tutorials, and learning materials for software engineers transitioning into artificial intelligence engineering roles. The project organizes resources that cover fundamental topics required to understand modern AI systems, including transformers, vector embeddings, tokenization, infrastructure design, and mixture-of-experts architectures. Instead of presenting isolated tutorials, the repository provides a structured pathway that guides engineers through the technical knowledge needed to build and deploy large language model systems. The materials include curated research papers, blog posts, and code examples that explain both theoretical foundations and practical implementation strategies. ... -
24
Ling
Ling is a MoE LLM provided and open-sourced by InclusionAI
...As more developers and researchers engage with the platform, we can expect rapid advancements and improvements, leading to even more sophisticated applications. Model inference and API code (e.g. integration with Transformers). This collaborative approach accelerates development and ensures that the models remain at the forefront of technology, addressing emerging challenges in various fields. -
25
InstantCharacter
Personalize Any Characters with a Scalable Diffusion Transformer
...It works by adapting a base image generation model with a lightweight adapter so that you can produce character-preserving generations in various downstream tasks (e.g. changing pose, clothing, scene) without needing full model fine-tuning. Works with huggingface/transformers/diffusers ecosystems.
