From NVIDIA Megatron-LM for visibility #18
+376,909
−35,102
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Remove extra barrier in checkpoint flow See merge request ADLR/megatron-lm!3626
Fix error when TE is not installed See merge request ADLR/megatron-lm!3625
…itializations and associated weight decay skipping.
Adding support for Spike No More embedding initializations and associated weight decay skipping. See merge request ADLR/megatron-lm!3500
MiMo video VLM train example See merge request ADLR/megatron-lm!3543
ci: Retry on `free(): invalid pointer` See merge request ADLR/megatron-lm!3632
Signed-off-by: oliver könig <okoenig@nvidia.com>
Co-authored-by: Keshav Santhanam <ksanthanam@nvidia.com> Co-authored-by: William Dykas <wdykas@cs-oci-ord-login-02.cm.cluster>
Add Dynamic Backend Inference Tests See merge request ADLR/megatron-lm!3475
…ate loading with PP>1 to ensure bit-wise match after saving and loading.
fix(distckpt, moe): Fix distckpt optimizer state loading with PP>1 to ensure bit-wise match after saving and loading. See merge request ADLR/megatron-lm!3394
tests: Fix segfaults (maybe?) See merge request ADLR/megatron-lm!3605
Co-authored-by: liu-zichen <liuzichen@dbis.nankai.edu.cn>
Fix mrope with context parallel See merge request ADLR/megatron-lm!3603
…rev_bwd_hidden_state_inputgrad for layer N-1
Bugfix: last layer N in backward should set prev_bwd_hidden_state_inputgrad for layer N-1 See merge request ADLR/megatron-lm!3645
Co-authored-by: yaoyu-33 <yaoyu.094@gmail.com>
fix(mla): Fix an issue with seq-packing + mla See merge request ADLR/megatron-lm!3467
…interface Co-authored-by: Mcore Bot <mcore-bot@nvidia.com> Co-authored-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: root <root@eos0502.eos.clusters.nvidia.com>
ModelCommProcessGroup integration into model interface See merge request ADLR/megatron-lm!3391
fix(mla): use mscale_all_dim for softmax_factor calculation See merge request ADLR/megatron-lm!2800
Use ruff linter See merge request ADLR/megatron-lm!3627
Remove FP8 calibration script See merge request ADLR/megatron-lm!3861
…d outside of the linear layer Co-authored-by: Chenjie Luo <chenjiel@nvidia.com>
Fix NEMO unittest where the weight is provided outside of the linear layer See merge request ADLR/megatron-lm!3828
add wandb_entity See merge request ADLR/megatron-lm!3864
…r DistOpt Co-authored-by: Slawek Kierat <skierat@nvidia.com> Co-authored-by: oliver könig <okoenig@nvidia.com> Co-authored-by: Mcore Bot <mcore-bot@nvidia.com>
Implement new optimizer checkpoint formats for DistOpt See merge request ADLR/megatron-lm!3532
build: Bump packaging See merge request ADLR/megatron-lm!3876
Signed-off-by: oliver könig <okoenig@nvidia.com>
Signed-off-by: oliver könig <okoenig@nvidia.com>
Signed-off-by: oliver könig <okoenig@nvidia.com>
Co-authored-by: Mcore Bot <mcore-bot@nvidia.com>
chore: Upgrade dependencies (2025-08-25) See merge request ADLR/megatron-lm!3887
Co-authored-by: Mcore Bot <mcore-bot@nvidia.com>
ci: Auto-publish megatron-fsdp See merge request ADLR/megatron-lm!3875
Co-authored-by: Mcore Bot <mcore-bot@nvidia.com> Co-authored-by: Jon Barker <jbarker@cw-pdx-cs-001-vscode-02.cm.cluster> Co-authored-by: Robert Kirby <rkirby@cw-dfw-cs-001-vscode-01.cm.cluster>
[1/4] Merge Megatron-RL into LM See merge request ADLR/megatron-lm!3646
…ze_model_parallel
Fix unsetting NCCL_COLLNET_ENABLE in initialize_model_parallel See merge request ADLR/megatron-lm!3884
…pping - (03) Support EP A2A overlap for interleaved PP and MTP Co-authored-by: Pingtian Li <pingtianl@nvidia.com>
feat(MoE): Support Expert Parallel A2A Overlapping - (03) Support EP A2A overlap for interleaved PP and MTP See merge request ADLR/megatron-lm!3074
Move cuda graph capture to core See merge request ADLR/megatron-lm!3782
Co-authored-by: Mcore Bot <mcore-bot@nvidia.com>
tests: Auto-validate weekly tests See merge request ADLR/megatron-lm!3764
Signed-off-by: oliver könig <okoenig@nvidia.com>
Signed-off-by: oliver könig <okoenig@nvidia.com>
Signed-off-by: oliver könig <okoenig@nvidia.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.