← Hackathons

MoE Gating (Linear + Softmax + TopK)

MoE Gating: bias-free linear router, full-expert stable softmax and top-k expert selection

Operator definition

logits = x @ w_router; p = softmax(logits, dim=-1); values, indices = stable_topk(p, k)

Interface

bf16_top_k(Tensor x, Tensor w_router, int k) -> (Tensor values, Tensor indices)

Data ranges

x:[B,H],w_router:[H,E],values,indices:[B,k]; B=1..1048576,H=1..65536,E=2..65536,k=1..min(4096,E)

Single-die stress scale

One physical die only (nominal 64 GiB HBM). Functional cases remain small and exhaustive; the independent three-tier stress suite exercises large bandwidth, compute and random-access workloads. Input + output tensors must total <=48 GiB, candidate workspace <=8 GiB; reserve >=8 GiB for runtime. Shape bounds are not a Cartesian guarantee: combinations exceeding the byte budget are invalid. Pressure correctness compares complete offline binary standards outside timing. Public stress runs are debugging only; formal pressure scoring starts only after all functional checks pass. Host wall time includes CPU work, dispatch and device synchronization, not only kernel time. No other die or CPU computation fallback.

Value ranges and coverage

BF16 x: [-32, 32]; w_router: [-4, 4]; FP32 probabilities: [0, 1]. Finite inputs only. Coverage includes zero, small magnitudes down to 2^-12, large magnitudes, positive/negative-only data, mixed scales, sparse data and representable boundaries. Task-specific tests also cover cancellation, BF16 ties and saturated softmax. NaN/Inf are excluded; subnormal preservation is not required. Not every tensor must span the entire range.

Parameters

Inputs

Outputs

Attrs

Numerical and implementation requirements

Compute logits x @ w_router in FP32 and softmax across all experts; output top-k probabilities as FP32 (atol 1e-6, rtol 2e-4). Rank by FP32 logits, equivalent to real-valued softmax order, so underflow does not create false ties. Exact logit ties prefer smaller expert IDs. Indices are exact int32 without duplicates. No bias or selected-probability renormalization; no intermediate BF16 rounding.

Outputs must remain on the input device, be contiguous, leave all inputs unchanged, and not alias any input. One physical die only; using the sibling die or multi-card communication is prohibited. Core computation, data movement and conversions must use custom NPU kernels. Built-in ACLNN operators, torch delegation, CPU fallback and cached answers are prohibited.

BF16 x: [-32, 32]; w_router: [-4, 4]; FP32 probabilities: [0, 1]. Finite inputs only. Coverage includes zero, small magnitudes down to 2^-12, large magnitudes, positive/negative-only data, mixed scales, sparse data and representable boundaries. Task-specific tests also cover cancellation, BF16 ties and saturated softmax. NaN/Inf are excluded; subnormal preservation is not required. Not every tensor must span the entire range.

One physical die only (nominal 64 GiB HBM). Functional cases remain small and exhaustive; the independent three-tier stress suite exercises large bandwidth, compute and random-access workloads. Input + output tensors must total <=48 GiB, candidate workspace <=8 GiB; reserve >=8 GiB for runtime. Shape bounds are not a Cartesian guarantee: combinations exceeding the byte budget are invalid. Pressure correctness compares complete offline binary standards outside timing. Public stress runs are debugging only; formal pressure scoring starts only after all functional checks pass. Host wall time includes CPU work, dispatch and device synchronization, not only kernel time. No other die or CPU computation fallback.

Scoring: accuracy 20%, performance 80%; compilation has no bonus. All accuracy checks must pass before pressure evaluation.