#moe

0 tools tagged moe, 1 news item

News tagged #moe

  1. Ai2

    Ai2 opens Olmo-core 3 for trillion parameter MoEs

    Olmo-core 3 replaces FSDP with GPU-resident distributed data parallelism for MoE training, hitting 52,000 tokens per second per GPU on eight B300s.