all AI news
[P] TensorRT-LLM Backend for WhisperS2T (~2x Speedup than CTranslate2)
Jan. 28, 2024, 4:49 p.m. | /u/Financial-Beach1587
Machine Learning www.reddit.com
**Added TensorRT-LLM Support:**
* \~ 2x Inference Speedup: WhisperS2T now supports the TensorRT-LLM backend, achieving double the inference speed compared to the CTranslate2 backend! The current optimal configuration on an A30 GPU achieves transcription of 1-hour files in approximately 18 seconds.
* As far as I know, this is the first proper implementation of TensorRT-LLM for Whisper with batching and …
a30 backend current hey inference llm machinelearning major openai speech speech-to-text speed support tensorrt tensorrt-llm text toolkit update whisper
More from www.reddit.com / Machine Learning
[D] Does DSPy actually change the LM weights?
1 day, 3 hours ago |
www.reddit.com
[D] Culture of Recycling Old Conference Submissions in ML
1 day, 6 hours ago |
www.reddit.com
Jobs in AI, ML, Big Data
Software Engineer for AI Training Data (School Specific)
@ G2i Inc | Remote
Software Engineer for AI Training Data (Python)
@ G2i Inc | Remote
Software Engineer for AI Training Data (Tier 2)
@ G2i Inc | Remote
Data Engineer
@ Lemon.io | Remote: Europe, LATAM, Canada, UK, Asia, Oceania
Artificial Intelligence – Bioinformatic Expert
@ University of Texas Medical Branch | Galveston, TX
Lead Developer (AI)
@ Cere Network | San Francisco, US