Benchmark Almanac

Sasha Rush

We track 3 benchmarks that Sasha Rush made or helped make.

0 in use 3 with public code 0 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
GPU Puzzles Models implement parallel GPU kernels for 14 puzzles. The port runs each kernel against a reference and rejects serial implementations. GPU kernels Jul 2022 90.5% Not in cards
LLM Training Puzzles Models solve eight coding puzzles on distributed neural network training, including optimizer state and pipeline parallelism. The hub evaluates each solution in a sandbox. Machine learning engineering Jun 2023 Not in cards
Transformer Puzzles (RASPy) Models complete eight sequence-programming puzzles in RASPy. The hub allows up to eight attempts per puzzle and checks solutions with executable tests. Function-level problems Nov 2022 Not in cards