ai-agent-book 精选快照（<2MB 代码与文档，来自 github.com/bojieli/ai-agent-book）
Build latest book artifacts / build (push) Canceled after 0s
dependency resolution / resolve (3.11) (push) Canceled after 0s
dependency resolution / resolve (3.13) (push) Canceled after 0s
deploy-pages / build (push) Canceled after 0s
deploy-pages / deploy (push) Canceled after 0s
i18n consistency check / check (push) Canceled after 0s
provider adoption tests / test (chapter2/context-compression) (push) Canceled after 0s
provider adoption tests / test (chapter2/prompt-injection) (push) Canceled after 0s
provider adoption tests / test (chapter2/system-hint) (push) Canceled after 0s
provider adoption tests / test (chapter3/log-sanitization) (push) Canceled after 0s
web-search-agent tests / test (push) Canceled after 0s
web-search-agent tests / agentbook (push) Canceled after 0s
Build latest book artifacts / build (push) Canceled after 0s
dependency resolution / resolve (3.11) (push) Canceled after 0s
dependency resolution / resolve (3.13) (push) Canceled after 0s
deploy-pages / build (push) Canceled after 0s
deploy-pages / deploy (push) Canceled after 0s
i18n consistency check / check (push) Canceled after 0s
provider adoption tests / test (chapter2/context-compression) (push) Canceled after 0s
provider adoption tests / test (chapter2/prompt-injection) (push) Canceled after 0s
provider adoption tests / test (chapter2/system-hint) (push) Canceled after 0s
provider adoption tests / test (chapter3/log-sanitization) (push) Canceled after 0s
web-search-agent tests / test (push) Canceled after 0s
web-search-agent tests / agentbook (push) Canceled after 0s
This commit is contained in:
@@ -0,0 +1,326 @@
|
||||
# Cursor Chat: ai-agent-book
|
||||
|
||||
## Metadata
|
||||
- **Project**: ai-agent-book
|
||||
- **Path**: `/Users/boj`
|
||||
- **Date**: 2025-10-01 12:31:22
|
||||
- **Session ID**: `2a1556e8-109d-4f11-8f86-614c97698c26`
|
||||
|
||||
## Conversation
|
||||
|
||||
### 👤 You
|
||||
|
||||
(TaskRunner pid=153792) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
|
||||
(TaskRunner pid=153792) 'all_ranks': False,
|
||||
(TaskRunner pid=153792) 'enable': False,
|
||||
(TaskRunner pid=153792) 'ranks': [],
|
||||
(TaskRunner pid=153792) 'save_path': 'outputs/profile',
|
||||
(TaskRunner pid=153792) 'tool': None,
|
||||
(TaskRunner pid=153792) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig',
|
||||
(TaskRunner pid=153792) 'analysis': True,
|
||||
(TaskRunner pid=153792) 'contents': [],
|
||||
(TaskRunner pid=153792) 'discrete': False,
|
||||
(TaskRunner pid=153792) 'level': 'level1'},
|
||||
(TaskRunner pid=153792) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
|
||||
(TaskRunner pid=153792) 'discrete': False},
|
||||
(TaskRunner pid=153792) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig',
|
||||
(TaskRunner pid=153792) 'step_end': None,
|
||||
(TaskRunner pid=153792) 'step_start': 0},
|
||||
(TaskRunner pid=153792) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig',
|
||||
(TaskRunner pid=153792) 'stack_depth': 32,
|
||||
(TaskRunner pid=153792) 'trace_alloc_max_entries': 100000}}},
|
||||
(TaskRunner pid=153792) 'prompt_length': 2048,
|
||||
(TaskRunner pid=153792) 'response_length': 16384,
|
||||
(TaskRunner pid=153792) 'skip_dump_dir': '/tmp/rollout_dump',
|
||||
(TaskRunner pid=153792) 'skip_rollout': False,
|
||||
(TaskRunner pid=153792) 'skip_tokenizer_init': True,
|
||||
(TaskRunner pid=153792) 'temperature': 1.0,
|
||||
(TaskRunner pid=153792) 'tensor_model_parallel_size': 4,
|
||||
(TaskRunner pid=153792) 'top_k': -1,
|
||||
(TaskRunner pid=153792) 'top_p': 1,
|
||||
(TaskRunner pid=153792) 'trace': {'_target_': 'verl.workers.config.TraceConfig',
|
||||
(TaskRunner pid=153792) 'backend': None,
|
||||
(TaskRunner pid=153792) 'token2text': False},
|
||||
(TaskRunner pid=153792) 'update_weights_bucket_megabytes': 512,
|
||||
(TaskRunner pid=153792) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig',
|
||||
(TaskRunner pid=153792) 'do_sample': False,
|
||||
(TaskRunner pid=153792) 'n': 30,
|
||||
(TaskRunner pid=153792) 'temperature': 1.0,
|
||||
(TaskRunner pid=153792) 'top_k': -1,
|
||||
(TaskRunner pid=153792) 'top_p': 0.6}}},
|
||||
(TaskRunner pid=153792) 'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig',
|
||||
(TaskRunner pid=153792) 'adv_estimator': 'grpo',
|
||||
(TaskRunner pid=153792) 'gamma': 1.0,
|
||||
(TaskRunner pid=153792) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig',
|
||||
(TaskRunner pid=153792) 'horizon': 10000,
|
||||
(TaskRunner pid=153792) 'kl_coef': 0.0,
|
||||
(TaskRunner pid=153792) 'target_kl': 0.1,
|
||||
(TaskRunner pid=153792) 'type': 'fixed'},
|
||||
(TaskRunner pid=153792) 'kl_penalty': 'kl',
|
||||
(TaskRunner pid=153792) 'lam': 1.0,
|
||||
(TaskRunner pid=153792) 'norm_adv_by_std_in_grpo': True,
|
||||
(TaskRunner pid=153792) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0},
|
||||
(TaskRunner pid=153792) 'use_kl_in_reward': False,
|
||||
(TaskRunner pid=153792) 'use_pf_ppo': False},
|
||||
(TaskRunner pid=153792) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig',
|
||||
(TaskRunner pid=153792) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig',
|
||||
(TaskRunner pid=153792) 'async_save': False,
|
||||
(TaskRunner pid=153792) 'load_contents': ['model', 'optimizer', 'extra'],
|
||||
(TaskRunner pid=153792) 'save_contents': ['model', 'optimizer', 'extra']},
|
||||
(TaskRunner pid=153792) 'cliprange_value': 0.5,
|
||||
(TaskRunner pid=153792) 'enable': None,
|
||||
(TaskRunner pid=153792) 'forward_max_token_len_per_gpu': 32768,
|
||||
(TaskRunner pid=153792) 'forward_micro_batch_size': None,
|
||||
(TaskRunner pid=153792) 'forward_micro_batch_size_per_gpu': None,
|
||||
(TaskRunner pid=153792) 'grad_clip': 1.0,
|
||||
(TaskRunner pid=153792) 'loss_agg_mode': 'token-mean',
|
||||
(TaskRunner pid=153792) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg',
|
||||
(TaskRunner pid=153792) 'enable_activation_offload': False,
|
||||
(TaskRunner pid=153792) 'enable_gradient_checkpointing': True,
|
||||
(TaskRunner pid=153792) 'external_lib': None,
|
||||
(TaskRunner pid=153792) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig',
|
||||
(TaskRunner pid=153792) 'entropy_checkpointing': False,
|
||||
(TaskRunner pid=153792) 'entropy_from_logits_with_chunking': False,
|
||||
(TaskRunner pid=153792) 'forward_only': False,
|
||||
(TaskRunner pid=153792) 'forward_prefetch': False,
|
||||
(TaskRunner pid=153792) 'fsdp_size': -1,
|
||||
(TaskRunner pid=153792) 'model_dtype': 'fp32',
|
||||
(TaskRunner pid=153792) 'offload_policy': False,
|
||||
(TaskRunner pid=153792) 'optimizer_offload': False,
|
||||
(TaskRunner pid=153792) 'param_offload': False,
|
||||
(TaskRunner pid=153792) 'reshard_after_forward': True,
|
||||
(TaskRunner pid=153792) 'strategy': 'fsdp',
|
||||
(TaskRunner pid=153792) 'ulysses_sequence_parallel_size': 1,
|
||||
(TaskRunner pid=153792) 'use_orig_params': False,
|
||||
(TaskRunner pid=153792) 'use_torch_compile': True,
|
||||
(TaskRunner pid=153792) 'wrap_policy': {'min_num_params': 0}},
|
||||
(TaskRunner pid=153792) 'lora_alpha': 16,
|
||||
(TaskRunner pid=153792) 'lora_rank': 0,
|
||||
(TaskRunner pid=153792) 'override_config': {},
|
||||
(TaskRunner pid=153792) 'path': '~/models/deepseek-llm-7b-chat',
|
||||
(TaskRunner pid=153792) 'target_modules': 'all-linear',
|
||||
(TaskRunner pid=153792) 'tokenizer_path': '/root/verl/recipe/retool/checkpoint/multiturn-sft-qwen-2.5-32b-instruct/global_step_372',
|
||||
(TaskRunner pid=153792) 'trust_remote_code': False,
|
||||
(TaskRunner pid=153792) 'use_remove_padding': False,
|
||||
(TaskRunner pid=153792) 'use_shm': False},
|
||||
(TaskRunner pid=153792) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig',
|
||||
(TaskRunner pid=153792) 'betas': [0.9, 0.999],
|
||||
(TaskRunner pid=153792) 'clip_grad': 1.0,
|
||||
(TaskRunner pid=153792) 'lr': 1e-05,
|
||||
(TaskRunner pid=153792) 'lr_warmup_steps': -1,
|
||||
(TaskRunner pid=153792) 'lr_warmup_steps_ratio': 0.0,
|
||||
(TaskRunner pid=153792) 'min_lr_ratio': 0.0,
|
||||
(TaskRunner pid=153792) 'num_cycles': 0.5,
|
||||
(TaskRunner pid=153792) 'total_training_steps': -1,
|
||||
(TaskRunner pid=153792) 'warmup_style': 'constant',
|
||||
(TaskRunner pid=153792) 'weight_decay': 0.01},
|
||||
(TaskRunner pid=153792) 'ppo_epochs': 1,
|
||||
(TaskRunner pid=153792) 'ppo_max_token_len_per_gpu': 32768,
|
||||
(TaskRunner pid=153792) 'ppo_micro_batch_size': None,
|
||||
(TaskRunner pid=153792) 'ppo_micro_batch_size_per_gpu': None,
|
||||
(TaskRunner pid=153792) 'ppo_mini_batch_size': 64,
|
||||
(TaskRunner pid=153792) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
|
||||
(TaskRunner pid=153792) 'all_ranks': False,
|
||||
(TaskRunner pid=153792) 'enable': False,
|
||||
(TaskRunner pid=153792) 'ranks': [],
|
||||
(TaskRunner pid=153792) 'save_path': 'outputs/profile',
|
||||
(TaskRunner pid=153792) 'tool': None,
|
||||
(TaskRunner pid=153792) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig',
|
||||
(TaskRunner pid=153792) 'analysis': True,
|
||||
(TaskRunner pid=153792) 'contents': [],
|
||||
(TaskRunner pid=153792) 'discrete': False,
|
||||
(TaskRunner pid=153792) 'level': 'level1'},
|
||||
(TaskRunner pid=153792) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
|
||||
(TaskRunner pid=153792) 'discrete': False},
|
||||
(TaskRunner pid=153792) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig',
|
||||
(TaskRunner pid=153792) 'step_end': None,
|
||||
(TaskRunner pid=153792) 'step_start': 0},
|
||||
(TaskRunner pid=153792) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig',
|
||||
(TaskRunner pid=153792) 'stack_depth': 32,
|
||||
(TaskRunner pid=153792) 'trace_alloc_max_entries': 100000}}},
|
||||
(TaskRunner pid=153792) 'rollout_n': 16,
|
||||
(TaskRunner pid=153792) 'shuffle': False,
|
||||
(TaskRunner pid=153792) 'strategy': 'fsdp',
|
||||
(TaskRunner pid=153792) 'ulysses_sequence_parallel_size': 1,
|
||||
(TaskRunner pid=153792) 'use_dynamic_bsz': True},
|
||||
(TaskRunner pid=153792) 'custom_reward_function': {'name': 'compute_score',
|
||||
(TaskRunner pid=153792) 'path': 'recipe/retool/retool.py'},
|
||||
(TaskRunner pid=153792) 'data': {'apply_chat_template_kwargs': {},
|
||||
(TaskRunner pid=153792) 'custom_cls': {'name': 'CustomRLHFDataset',
|
||||
(TaskRunner pid=153792) 'path': 'recipe/retool/retool.py'},
|
||||
(TaskRunner pid=153792) 'datagen': {'name': None, 'path': None},
|
||||
(TaskRunner pid=153792) 'dataloader_num_workers': 8,
|
||||
(TaskRunner pid=153792) 'filter_overlong_prompts': True,
|
||||
(TaskRunner pid=153792) 'filter_overlong_prompts_workers': 1,
|
||||
(TaskRunner pid=153792) 'image_key': 'images',
|
||||
(TaskRunner pid=153792) 'max_prompt_length': 2048,
|
||||
(TaskRunner pid=153792) 'max_response_length': 16384,
|
||||
(TaskRunner pid=153792) 'prompt_key': 'prompt',
|
||||
(TaskRunner pid=153792) 'return_full_prompt': False,
|
||||
(TaskRunner pid=153792) 'return_multi_modal_inputs': True,
|
||||
(TaskRunner pid=153792) 'return_raw_chat': True,
|
||||
(TaskRunner pid=153792) 'return_raw_input_ids': False,
|
||||
(TaskRunner pid=153792) 'reward_fn_key': 'data_source',
|
||||
(TaskRunner pid=153792) 'sampler': {'class_name': None, 'class_path': None},
|
||||
(TaskRunner pid=153792) 'shuffle': True,
|
||||
(TaskRunner pid=153792) 'tokenizer': None,
|
||||
(TaskRunner pid=153792) 'train_batch_size': 512,
|
||||
(TaskRunner pid=153792) 'train_files': ['/dataset/BytedTsinghua-SIA/DAPO-Math-17k'],
|
||||
(TaskRunner pid=153792) 'truncation': 'error',
|
||||
(TaskRunner pid=153792) 'trust_remote_code': False,
|
||||
(TaskRunner pid=153792) 'use_shm': False,
|
||||
(TaskRunner pid=153792) 'val_batch_size': None,
|
||||
(TaskRunner pid=153792) 'val_files': ['/dataset/yentinglin/aime_2025'],
|
||||
(TaskRunner pid=153792) 'validation_shuffle': False,
|
||||
(TaskRunner pid=153792) 'video_key': 'videos'},
|
||||
(TaskRunner pid=153792) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
|
||||
(TaskRunner pid=153792) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
|
||||
(TaskRunner pid=153792) 'controller_nsight_options': {'cuda-graph-trace': 'graph',
|
||||
(TaskRunner pid=153792) 'cuda-memory-usage': 'true',
|
||||
(TaskRunner pid=153792) 'trace': 'cuda,nvtx,cublas,ucx'},
|
||||
(TaskRunner pid=153792) 'discrete': False,
|
||||
(TaskRunner pid=153792) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi',
|
||||
(TaskRunner pid=153792) 'capture-range-end': None,
|
||||
(TaskRunner pid=153792) 'cuda-graph-trace': 'graph',
|
||||
(TaskRunner pid=153792) 'cuda-memory-usage': 'true',
|
||||
(TaskRunner pid=153792) 'kill': 'none',
|
||||
(TaskRunner pid=153792) 'trace': 'cuda,nvtx,cublas,ucx'}},
|
||||
(TaskRunner pid=153792) 'torch_memory': {'context': 'all',
|
||||
(TaskRunner pid=153792) 'kw_args': {},
|
||||
(TaskRunner pid=153792) 'stack_depth': 32,
|
||||
(TaskRunner pid=153792) 'stacks': 'all',
|
||||
(TaskRunner pid=153792) 'trace_alloc_max_entries': 100000}},
|
||||
(TaskRunner pid=153792) 'profile_continuous_steps': False,
|
||||
(TaskRunner pid=153792) 'save_path': 'outputs/profile',
|
||||
(TaskRunner pid=153792) 'steps': None,
|
||||
(TaskRunner pid=153792) 'tool': None},
|
||||
(TaskRunner pid=153792) 'ray_kwargs': {'ray_init': {'num_cpus': None}, 'timeline_json_file': None},
|
||||
(TaskRunner pid=153792) 'reward_model': {'enable': False,
|
||||
(TaskRunner pid=153792) 'enable_resource_pool': False,
|
||||
(TaskRunner pid=153792) 'forward_max_token_len_per_gpu': 32768,
|
||||
(TaskRunner pid=153792) 'launch_reward_fn_async': False,
|
||||
(TaskRunner pid=153792) 'max_length': None,
|
||||
(TaskRunner pid=153792) 'micro_batch_size': None,
|
||||
(TaskRunner pid=153792) 'micro_batch_size_per_gpu': None,
|
||||
(TaskRunner pid=153792) 'model': {'external_lib': None,
|
||||
(TaskRunner pid=153792) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig',
|
||||
(TaskRunner pid=153792) 'forward_prefetch': False,
|
||||
(TaskRunner pid=153792) 'fsdp_size': -1,
|
||||
(TaskRunner pid=153792) 'param_offload': False,
|
||||
(TaskRunner pid=153792) 'reshard_after_forward': True,
|
||||
(TaskRunner pid=153792) 'wrap_policy': {'min_num_params': 0}},
|
||||
(TaskRunner pid=153792) 'input_tokenizer': '/root/verl/recipe/retool/checkpoint/multiturn-sft-qwen-2.5-32b-instruct/global_step_372',
|
||||
(TaskRunner pid=153792) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1',
|
||||
(TaskRunner pid=153792) 'trust_remote_code': False,
|
||||
(TaskRunner pid=153792) 'use_fused_kernels': False,
|
||||
(TaskRunner pid=153792) 'use_remove_padding': False,
|
||||
(TaskRunner pid=153792) 'use_shm': False},
|
||||
(TaskRunner pid=153792) 'n_gpus_per_node': 0,
|
||||
(TaskRunner pid=153792) 'nnodes': 0,
|
||||
(TaskRunner pid=153792) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
|
||||
(TaskRunner pid=153792) 'all_ranks': False,
|
||||
(TaskRunner pid=153792) 'enable': False,
|
||||
(TaskRunner pid=153792) 'ranks': [],
|
||||
(TaskRunner pid=153792) 'save_path': 'outputs/profile',
|
||||
(TaskRunner pid=153792) 'tool': None,
|
||||
(TaskRunner pid=153792) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig',
|
||||
(TaskRunner pid=153792) 'analysis': True,
|
||||
(TaskRunner pid=153792) 'contents': [],
|
||||
(TaskRunner pid=153792) 'discrete': False,
|
||||
(TaskRunner pid=153792) 'level': 'level1'},
|
||||
(TaskRunner pid=153792) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
|
||||
(TaskRunner pid=153792) 'discrete': False},
|
||||
(TaskRunner pid=153792) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig',
|
||||
(TaskRunner pid=153792) 'step_end': None,
|
||||
(TaskRunner pid=153792) 'step_start': 0},
|
||||
(TaskRunner pid=153792) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig',
|
||||
(TaskRunner pid=153792) 'stack_depth': 32,
|
||||
(TaskRunner pid=153792) 'trace_alloc_max_entries': 100000}}},
|
||||
(TaskRunner pid=153792) 'reward_manager': 'naive',
|
||||
(TaskRunner pid=153792) 'sandbox_fusion': {'max_concurrent': 64,
|
||||
(TaskRunner pid=153792) 'memory_limit_mb': 1024,
|
||||
(TaskRunner pid=153792) 'url': None},
|
||||
(TaskRunner pid=153792) 'strategy': 'fsdp',
|
||||
(TaskRunner pid=153792) 'ulysses_sequence_parallel_size': 1,
|
||||
(TaskRunner pid=153792) 'use_dynamic_bsz': True},
|
||||
(TaskRunner pid=153792) 'trainer': {'balance_batch': True,
|
||||
(TaskRunner pid=153792) 'critic_warmup': 0,
|
||||
(TaskRunner pid=153792) 'default_hdfs_dir': None,
|
||||
(TaskRunner pid=153792) 'default_local_dir': '/root/verl/recipe/retool/checkpoint/qwen2.5-32b_dapo',
|
||||
(TaskRunner pid=153792) 'del_local_ckpt_after_load': False,
|
||||
(TaskRunner pid=153792) 'device': 'cuda',
|
||||
(TaskRunner pid=153792) 'esi_redundant_time': 0,
|
||||
(TaskRunner pid=153792) 'experiment_name': 'qwen2.5-32b_dapo',
|
||||
(TaskRunner pid=153792) 'log_val_generations': 100,
|
||||
(TaskRunner pid=153792) 'logger': ['console', 'wandb'],
|
||||
(TaskRunner pid=153792) 'max_actor_ckpt_to_keep': None,
|
||||
(TaskRunner pid=153792) 'max_critic_ckpt_to_keep': None,
|
||||
(TaskRunner pid=153792) 'n_gpus_per_node': 8,
|
||||
(TaskRunner pid=153792) 'nnodes': 1,
|
||||
(TaskRunner pid=153792) 'project_name': 'boj_retool',
|
||||
(TaskRunner pid=153792) 'ray_wait_register_center_timeout': 300,
|
||||
(TaskRunner pid=153792) 'resume_from_path': None,
|
||||
(TaskRunner pid=153792) 'resume_mode': 'auto',
|
||||
(TaskRunner pid=153792) 'rollout_data_dir': None,
|
||||
(TaskRunner pid=153792) 'save_freq': 30,
|
||||
(TaskRunner pid=153792) 'test_freq': 5,
|
||||
(TaskRunner pid=153792) 'total_epochs': 1,
|
||||
(TaskRunner pid=153792) 'total_training_steps': None,
|
||||
(TaskRunner pid=153792) 'use_legacy_worker_impl': 'auto',
|
||||
(TaskRunner pid=153792) 'val_before_train': True,
|
||||
(TaskRunner pid=153792) 'val_only': False,
|
||||
(TaskRunner pid=153792) 'validation_data_dir': None}}
|
||||
(TaskRunner pid=153792) /root/verl/verl/trainer/main_ppo.py:264: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True
|
||||
(TaskRunner pid=153792) use_critic=need_critic(config),
|
||||
Error executing job with overrides: ['algorithm.adv_estimator=grpo', 'algorithm.use_kl_in_reward=False', 'algorithm.kl_ctrl.kl_coef=0.0', "data.train_files=['/dataset/BytedTsinghua-SIA/DAPO-Math-17k']", "data.val_files=['/dataset/yentinglin/aime_2025']", 'data.return_raw_chat=True', 'data.train_batch_size=512', 'data.max_prompt_length=2048', 'data.max_response_length=16384', 'data.filter_overlong_prompts=True', 'data.truncation=error', 'data.custom_cls.path=recipe/retool/retool.py', 'data.custom_cls.name=CustomRLHFDataset', 'custom_reward_function.path=recipe/retool/retool.py', 'custom_reward_function.name=compute_score', 'actor_rollout_ref.model.path=/root/verl/recipe/retool/checkpoint/multiturn-sft-qwen-2.5-32b-instruct/global_step_372', 'actor_rollout_ref.model.use_remove_padding=True', 'actor_rollout_ref.model.enable_gradient_checkpointing=True', 'actor_rollout_ref.actor.use_kl_loss=False', 'actor_rollout_ref.actor.kl_loss_coef=0.0', 'actor_rollout_ref.actor.clip_ratio_low=0.2', 'actor_rollout_ref.actor.clip_ratio_high=0.28', 'actor_rollout_ref.actor.clip_ratio_c=10.0', 'actor_rollout_ref.actor.optim.lr=1e-6', 'actor_rollout_ref.actor.use_dynamic_bsz=True', 'actor_rollout_ref.actor.ppo_mini_batch_size=64', 'actor_rollout_ref.actor.ppo_max_token_len_per_gpu=18432', 'actor_rollout_ref.actor.ulysses_sequence_parallel_size=8', 'actor_rollout_ref.actor.fsdp_config.param_offload=True', 'actor_rollout_ref.actor.fsdp_config.optimizer_offload=True', 'actor_rollout_ref.ref.log_prob_max_token_len_per_gpu=73728', 'actor_rollout_ref.rollout.name=vllm', 'actor_rollout_ref.rollout.mode=async', 'actor_rollout_ref.rollout.tensor_model_parallel_size=4', 'actor_rollout_ref.rollout.multi_turn.enable=True', 'actor_rollout_ref.rollout.multi_turn.max_user_turns=8', 'actor_rollout_ref.rollout.multi_turn.max_assistant_turns=8', 'actor_rollout_ref.rollout.multi_turn.tool_config_path=recipe/retool/sandbox_fusion_tool_config.yaml', 'actor_rollout_ref.rollout.multi_turn.format=hermes', 'actor_rollout_ref.rollout.gpu_memory_utilization=0.9', 'actor_rollout_ref.rollout.n=16', 'actor_rollout_ref.rollout.val_kwargs.top_p=0.6', 'actor_rollout_ref.rollout.val_kwargs.temperature=1.0', 'actor_rollout_ref.rollout.val_kwargs.n=30', 'trainer.logger=[console,wandb]', 'trainer.project_name=boj_retool', 'trainer.experiment_name=qwen2.5-32b_dapo', 'trainer.n_gpus_per_node=8', 'trainer.val_before_train=True', 'trainer.log_val_generations=100', 'trainer.nnodes=1', 'trainer.save_freq=30', 'trainer.default_local_dir=/root/verl/recipe/retool/checkpoint/qwen2.5-32b_dapo', 'trainer.test_freq=5', 'trainer.total_epochs=1']
|
||||
Traceback (most recent call last):
|
||||
File "/root/verl/verl/trainer/main_ppo.py", line 42, in main
|
||||
run_ppo(config)
|
||||
~~~~~~~^^^^^^^^
|
||||
File "/root/verl/verl/trainer/main_ppo.py", line 85, in run_ppo
|
||||
ray.get(runner.run.remote(config))
|
||||
~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/root/miniconda3/lib/python3.13/site-packages/ray/_private/auto_init_hook.py", line 22, in auto_init_wrapper
|
||||
return fn(*args, **kwargs)
|
||||
File "/root/miniconda3/lib/python3.13/site-packages/ray/_private/client_mode_hook.py", line 104, in wrapper
|
||||
return func(*args, **kwargs)
|
||||
File "/root/miniconda3/lib/python3.13/site-packages/ray/_private/worker.py", line 2882, in get
|
||||
values, debugger_breakpoint = worker.get_objects(object_refs, timeout=timeout)
|
||||
~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/root/miniconda3/lib/python3.13/site-packages/ray/_private/worker.py", line 968, in get_objects
|
||||
raise value.as_instanceof_cause()
|
||||
ray.exceptions.RayTaskError(ValueError): ray::TaskRunner.run() (pid=153792, ip=172.17.0.2, actor_id=a3b627133447f0466782aff001000000, repr=<main_ppo.TaskRunner object at 0x7f6615317cb0>)
|
||||
File "/root/verl/verl/trainer/main_ppo.py", line 277, in run
|
||||
tokenizer = hf_tokenizer(local_path, trust_remote_code=trust_remote_code)
|
||||
File "/root/verl/verl/utils/tokenizer.py", line 60, in hf_tokenizer
|
||||
tokenizer = AutoTokenizer.from_pretrained(name_or_path, **kwargs)
|
||||
File "/root/miniconda3/lib/python3.13/site-packages/transformers/models/auto/tokenization_auto.py", line 1078, in from_pretrained
|
||||
config = AutoConfig.from_pretrained(
|
||||
pretrained_model_name_or_path, trust_remote_code=trust_remote_code, **kwargs
|
||||
)
|
||||
File "/root/miniconda3/lib/python3.13/site-packages/transformers/models/auto/configuration_auto.py", line 1329, in from_pretrained
|
||||
raise ValueError(
|
||||
...<3 lines>...
|
||||
)
|
||||
ValueError: Unrecognized model in /root/verl/recipe/retool/checkpoint/multiturn-sft-qwen-2.5-32b-instruct/global_step_372. Should have a `model_type` key in its config.json, or contain one of the following strings in its name: aimv2, aimv2_vision_model, albert, align, altclip, apertus, arcee, aria, aria_text, audio-spectrogram-transformer, autoformer, aya_vision, bamba, bark, bart, beit, bert, bert-generation, big_bird, bigbird_pegasus, biogpt, bit, bitnet, blenderbot, blenderbot-small, blip, blip-2, blip_2_qformer, bloom, bridgetower, bros, camembert, canine, chameleon, chinese_clip, chinese_clip_vision_model, clap, clip, clip_text_model, clip_vision_model, clipseg, clvp, code_llama, codegen, cohere, cohere2, cohere2_vision, colpali, colqwen2, conditional_detr, convbert, convnext, convnextv2, cpmant, csm, ctrl, cvt, d_fine, dab-detr, dac, data2vec-audio, data2vec-text, data2vec-vision, dbrx, deberta, deberta-v2, decision_transformer, deepseek_v2, deepseek_v3, deepseek_vl, deepseek_vl_hybrid, deformable_detr, deit, depth_anything, depth_pro, deta, detr, dia, diffllama, dinat, dinov2, dinov2_with_registers, dinov3_convnext, dinov3_vit, distilbert, doge, donut-swin, dots1, dpr, dpt, efficientformer, efficientloftr, efficientnet, electra, emu3, encodec, encoder-decoder, eomt, ernie, ernie4_5, ernie4_5_moe, ernie_m, esm, evolla, exaone4, falcon, falcon_h1, falcon_mamba, fastspeech2_conformer, fastspeech2_conformer_with_hifigan, flaubert, flava, florence2, fnet, focalnet, fsmt, funnel, fuyu, gemma, gemma2, gemma3, gemma3_text, gemma3n, gemma3n_audio, gemma3n_text, gemma3n_vision, git, glm, glm4, glm4_moe, glm4v, glm4v_moe, glm4v_moe_text, glm4v_text, glpn, got_ocr2, gpt-sw3, gpt2, gpt_bigcode, gpt_neo, gpt_neox, gpt_neox_japanese, gpt_oss, gptj, gptsan-japanese, granite, granite_speech, granitemoe, granitemoehybrid, granitemoeshared, granitevision, graphormer, grounding-dino, groupvit, helium, hgnet_v2, hiera, hubert, hunyuan_v1_dense, hunyuan_v1_moe, ibert, idefics, idefics2, idefics3, idefics3_vision, ijepa, imagegpt, informer, instructblip, instructblipvideo, internvl, internvl_vision, jamba, janus, jetmoe, jukebox, kosmos-2, kosmos-2.5, kyutai_speech_to_text, layoutlm, layoutlmv2, layoutlmv3, led, levit, lfm2, lightglue, lilt, llama, llama4, llama4_text, llava, llava_next, llava_next_video, llava_onevision, longformer, longt5, luke, lxmert, m2m_100, mamba, mamba2, marian, markuplm, mask2former, maskformer, maskformer-swin, mbart, mctct, mega, megatron-bert, metaclip_2, mgp-str, mimi, minimax, mistral, mistral3, mixtral, mlcd, mllama, mm-grounding-dino, mobilebert, mobilenet_v1, mobilenet_v2, mobilevit, mobilevitv2, modernbert, modernbert-decoder, moonshine, moshi, mpnet, mpt, mra, mt5, musicgen, musicgen_melody, mvp, nat, nemotron, nezha, nllb-moe, nougat, nystromformer, olmo, olmo2, olmoe, omdet-turbo, oneformer, open-llama, openai-gpt, opt, ovis2, owlv2, owlvit, paligemma, patchtsmixer, patchtst, pegasus, pegasus_x, perceiver, perception_encoder, perception_lm, persimmon, phi, phi3, phi4_multimodal, phimoe, pix2struct, pixtral, plbart, poolformer, pop2piano, prompt_depth_anything, prophetnet, pvt, pvt_v2, qdqbert, qwen2, qwen2_5_omni, qwen2_5_vl, qwen2_5_vl_text, qwen2_audio, qwen2_audio_encoder, qwen2_moe, qwen2_vl, qwen2_vl_text, qwen3, qwen3_moe, rag, realm, recurrent_gemma, reformer, regnet, rembert, resnet, retribert, roberta, roberta-prelayernorm, roc_bert, roformer, rt_detr, rt_detr_resnet, rt_detr_v2, rwkv, sam, sam2, sam2_hiera_det_model, sam2_video, sam2_vision_model, sam_hq, sam_hq_vision_model, sam_vision_model, seamless_m4t, seamless_m4t_v2, seed_oss, segformer, seggpt, sew, sew-d, shieldgemma2, siglip, siglip2, siglip_vision_model, smollm3, smolvlm, smolvlm_vision, speech-encoder-decoder, speech_to_text, speech_to_text_2, speecht5, splinter, squeezebert, stablelm, starcoder2, superglue, superpoint, swiftformer, swin, swin2sr, swinv2, switch_transformers, t5, t5gemma, table-transformer, tapas, textnet, time_series_transformer, timesfm, timesformer, timm_backbone, timm_wrapper, trajectory_transformer, transfo-xl, trocr, tvlt, tvp, udop, umt5, unispeech, unispeech-sat, univnet, upernet, van, video_llava, videomae, vilt, vipllava, vision-encoder-decoder, vision-text-dual-encoder, visual_bert, vit, vit_hybrid, vit_mae, vit_msn, vitdet, vitmatte, vitpose, vitpose_backbone, vits, vivit, vjepa2, voxtral, voxtral_encoder, wav2vec2, wav2vec2-bert, wav2vec2-conformer, wavlm, whisper, xclip, xcodec, xglm, xlm, xlm-prophetnet, xlm-roberta, xlm-roberta-xl, xlnet, xlstm, xmod, yolos, yoso, zamba, zamba2, zoedepth
|
||||
|
||||
---
|
||||
|
||||
### 🤖 Cursor Assistant
|
||||
|
||||
Looking at this error from your VERL training run, the issue is clear: the checkpoint at `/root/verl/recipe/retool/checkpoint/multiturn-sft-qwen-2.5-32b-instruct/global_step_372` is missing the `model_type` key in its `config.json` file, which the HuggingFace transformers library requires to identify the model architecture.
|
||||
|
||||
The problem occurs when the training process tries to load the tokenizer from your checkpoint. Based on the path name indicating this is a Qwen 2.5 32B model, the `config.json` in that checkpoint directory needs to have `"model_type": "qwen2"` specified.
|
||||
|
||||
**The root cause**: When the checkpoint was saved at global step 372, either the configuration wasn't properly serialized, or the checkpoint was created through a process that didn't preserve the model type metadata.
|
||||
|
||||
**The solution**: You need to manually add the model_type to the config.json file in that checkpoint directory. Open the file at `/root/verl/recipe/retool/checkpoint/multiturn-sft-qwen-2.5-32b-instruct/global_step_372/config.json` and ensure it contains a `"model_type": "qwen2"` entry at the top level of the JSON structure.
|
||||
|
||||
Alternatively, if you have the original model this checkpoint was derived from, you could copy the config.json from the original model directory and ensure all the other configuration parameters match your checkpoint's architecture.
|
||||
|
||||
Is this error blocking your current work on the book, or are you looking to document this type of issue in one of your chapters?
|
||||
|
||||
---
|
||||
|
||||
|
||||
*Exported from [Cursor View](https://github.com/saharmor/cursor-view)*
|
||||
Reference in New Issue
Block a user