Article
1
Kshitij Thakkar PRO
kshitijthakkar
AI & ML interests
AI observability + MoE efficiency engineer. Building tools that make GenAI traceable, measurable, and production-ready.
Recent Activity
published
an
article
about 19 hours ago
Scaling Mixture of Experts: Architecture Search for Billion-Parameter Language Models
updated
a collection
about 19 hours ago
Large MoE Architecture Search (1B-2B)
updated
a dataset
about 20 hours ago
kshitijthakkar/large-moe-inference-benchmark
Organizations
Articles 5
Article
1
Systematic Architecture Search for Mobile-Optimized Mixture of Experts Language Models
Systematic search for 1B-2B MoE models. Best: bs=1, ctx=2048 achieves 0.32 loss. Top-8 routing beats top-2.
-
kshitijthakkar/moe-1083m-781m-16x8-8L-large-moe-1.3b-bs4-ctx1024
Updated • 21 -
kshitijthakkar/moe-1083m-781m-16x8-8L-large-moe-1.3b-bs2-ctx2048
Updated • 22 -
kshitijthakkar/moe-1083m-781m-16x8-8L-large-moe-1.3b-bs2-ctx1024
Updated • 18 -
kshitijthakkar/moe-1083m-781m-16x8-8L-large-moe-1.3b-bs1-ctx2048
Updated • 27
Systematic search for 1B-2B MoE models. Best: bs=1, ctx=2048 achieves 0.32 loss. Top-8 routing beats top-2.
-
kshitijthakkar/moe-1083m-781m-16x8-8L-large-moe-1.3b-bs4-ctx1024
Updated • 21 -
kshitijthakkar/moe-1083m-781m-16x8-8L-large-moe-1.3b-bs2-ctx2048
Updated • 22 -
kshitijthakkar/moe-1083m-781m-16x8-8L-large-moe-1.3b-bs2-ctx1024
Updated • 18 -
kshitijthakkar/moe-1083m-781m-16x8-8L-large-moe-1.3b-bs1-ctx2048
Updated • 27
32 MoE models from 41 experiments exploring expert count, routing, and learning rates for mobile deployment.
spaces
11
Sleeping
Loggenix MoE 0.4B-A0.2B Demo
🧠
Running
1
E-Commerce Product Content Generator
🛒
Generate complete e-commerce product content from a description
Running
1
Multimodal Content Pipeline
🖼
Running
1
AI Content Creation Pipeline
🎨
Runtime error
2
Loggenix Moe 0.3B A0.1B Demo
🏢
Demo Space for my model loggenix-moe-0.3B-A0.1B
Sleeping
my_Akatsuki_theme
🚀
Preview and customize `my_Akatsuki_theme` in a Gradio interface
models
107
kshitijthakkar/moe-1083m-781m-16x8-8L-large-moe-1.3b-bs4-ctx1024
Updated
•
21
kshitijthakkar/moe-1083m-781m-16x8-8L-large-moe-1.3b-bs2-ctx2048
Updated
•
22
kshitijthakkar/moe-1083m-781m-16x8-8L-large-moe-1.3b-bs2-ctx1024
Updated
•
18
kshitijthakkar/moe-1083m-781m-16x8-8L-large-moe-1.3b-bs1-ctx2048
Updated
•
27
kshitijthakkar/moe-1083m-781m-16x8-8L-large-moe-1.3b-bs1-ctx1024
Updated
•
15
kshitijthakkar/moe-1083m-781m-16x8-8L-large-moe-1.3b-lr1e-03
Updated
•
20
kshitijthakkar/moe-1083m-781m-16x8-8L-large-moe-1.3b-lr5e-04
Updated
•
27
kshitijthakkar/moe-1083m-781m-16x8-8L-large-moe-1.3b-lr3e-04
Updated
•
20
kshitijthakkar/moe-1083m-781m-16x8-8L-large-moe-1.3b-lr2e-04
Updated
•
18
kshitijthakkar/moe-1083m-781m-16x8-8L-large-moe-1.3b-lr1e-04
Updated
•
28
datasets
378
kshitijthakkar/large-moe-inference-benchmark
Viewer
•
Updated
•
38
kshitijthakkar/loggenix-0.4B-flagged-responses
Viewer
•
Updated
•
4
•
21
kshitijthakkar/moe-inference-benchmark
Viewer
•
Updated
•
256
•
29
kshitijthakkar/nemotron-sft-balanced-stage1-2-ChatML-small
Viewer
•
Updated
•
25k
•
25
kshitijthakkar/nemotron-pre-training-balanced-250m
Viewer
•
Updated
•
113k
•
23
kshitijthakkar/nemotron-sft-balanced-stage1-2-ChatML-V2
Viewer
•
Updated
•
500k
•
24
kshitijthakkar/nemotron-sft-general-focused-stage1-2-ChatML-V3
Viewer
•
Updated
•
493k
•
16
kshitijthakkar/smoltrace-leaderboard
Viewer
•
Updated
•
100
•
93
kshitijthakkar/smoltrace-metrics-20260130_072640
Viewer
•
Updated
•
533
•
6
kshitijthakkar/smoltrace-traces-20260130_072640
Viewer
•
Updated
•
15
•
10