Create a model configuration by users

[Luciennnnnnn]

Hello, I am trying to use a SwinTransformer model with the configuration 'swin_base_patch4_window6_192'. Unfortunately, the timm library does not provide this configuration.

I am consistently using the timm.create_model interface and would like to define my own model configuration named 'swin_base_patch4_window6_192' in my project. This way, I can create the model using the timm.create_model interface.

Could you please provide guidance on how I can define this custom configuration in my project and use it with the timm.create_model function?

[Luciennnnnnn]

I have tried following code in my project:

@register_model
def swin_base_patch4_window6_192(pretrained=False, **kwargs):
    """ Swin-B @ 192x192
    """
    model_kwargs = dict(
        patch_size=4, window_size=6, embed_dim=128, depths=(2, 2, 18, 2), num_heads=(4, 8, 16, 32), **kwargs)
    return _create_swin_transformer('swin_base_patch4_window6_192', pretrained=pretrained, **model_kwargs)

However, it seems lossing some essential parts like default mean and std, or others...

[rwightman]

@LuoXin-s for dataset / pretrained related details you need to setup default pretrained cfgs for the new model, as these can vary with each dataset the model is trained on.

https://github.com/huggingface/pytorch-image-models/blob/main/timm/models/swin_transformer.py#L614-L698

[Luciennnnnnn]

@rwightman I added below code:

from timm.models.swin_transformer import _cfg

generate_default_cfgs({
    'swin_base_patch4_window6_192': _cfg()
})

But this seems not work properly. I only want to define model's hyperparameter, and no pretrained models will be loaded.

Could you please provide me a simple code snippet? Thanks!

[rwightman]

@LuoXin-s not sure what the actual goal is, if it's not to share pretrained weights or add a new def for others, etc but just to train that model for research you can skip the default cfg generation and just embed the pretrained cfg in the entry fn...

For the more permanent approach, the output of generate_default_cfgs needs to be assigned to a module level 'default_cfgs' var in the same module that the @register_model decorated entry fn exists (like all other models)

@register_model
def swin_base_patch4_window6_192(pretrained=False, **kwargs):
    """ Swin-B @ 192x192
    """
    default_cfg = kwargs.pop('pretrained_cfg', None) or _cfg(input_size=(3, 192, 192), pool_size=(6, 6))
    model_kwargs = dict(
        patch_size=4, window_size=6, embed_dim=128, depths=(2, 2, 18, 2), num_heads=(4, 8, 16, 32), **kwargs)
    return _create_swin_transformer(
        'swin_base_patch4_window6_192',
        pretrained_cfg=default_cfg,
        pretrained=pretrained,
        **model_kwargs,
    )

[Luciennnnnnn]

@rwightman Thanks for your kind instruction. One further problem is that I rely on timm.models.get_pretrained_cfg(model_name) to get corresponding mean and std, this is fine when I consistently use pretrained models. However, when I use model with random parameters, timm.models.get_pretrained_cfg(model_name) would return a None cfg. What should I do?

Is timm.create_model(model_name).default_cfg() or timm.create_model(model_name).pretrained_cfg() reliable for both cases?

[rwightman]

@LuoXin-s moving out of issues since not a bug..

you need to register model & pretrained configs properly to use models.get_pretrained_cfg() ... as mentioned, in the same file, follow existing model as an example (https://github.com/huggingface/pytorch-image-models/blob/main/timm/models/vision_transformer_hybrid.py#L102-L202 ..)

I don't know why you'd create a model and then destroy it by doing timm.create_model(model_name).pretrained_cfg() just to get the config. Every model created by the factory has the pretrained_cfg attribute (default_cfg is a deprecated name for that), there is no point in creating a model and discarding it, just get the config from the model you are actually going to use model.pretrained_cfg

There are helpers that will take the pretrained config from the model (or via argument / dict overrides), merge them all and output a final dict that can be passed via kwargs to downstream create loader/transform fns

ie https://github.com/huggingface/pytorch-image-models/blob/main/timm/data/config.py#L101

final_data_cfg_dict = resolve_model_data_config(
  model,
  args=  # optional argparse overrides
  pretrained_cfg= # optional dict overrides
)