Reference-free human-object
interaction editing

1Nanyang Technological University,  2Sun Yat-sen University,  3Nanjing Forestry University,  4Universiti Malaya,  5VinUniversity
*Corresponding authors.

TL;DR: We enables zero-shot human-object interaction edit

Previously available as a preprint titled “InteractEdit: Zero-Shot Editing of Human-Object Interactions in Images”.

Sample results of editing Human-Object Interaction in the source image (left). Existing methods overly preserve the structure, making interaction edits ineffective.

Qualitative Results

1. Sample of Interaction Edits

Orange indicates target interaction.

dribble ball

hold skateboard

2. Editing Interactions Compared to Other Methods

Orange indicates target interaction.

Qualitative results

Abstract

This paper presents InteractEdit, a novel framework for reference-free Human-Object Interaction (HOI) editing that tackles the challenging task of transforming an existing interaction in an image into a new, desired interaction while preserving the identities of the subject and object. Unlike prior image editing tasks such as attribute manipulation, object replacement or style transfer, HOI editing involves complex spatial, contextual, and relational dependencies inherent in HOI. Existing methods often overfit to the source image structure, limiting adaptability to the substantial structural modifications demanded by the new interactions. To address this, InteractEdit disassembles each scene into subject, object, and background components to disentangle intricate HOI relationship, and introduces a selective inversion strategy combined with Selective-Rank Adaptation (SeRA) to leverage pretrained interaction priors while learning visual identity from the source image. This enables a balanced trade-off between interaction editing and identity preservation. We also introduce IEBench, a new benchmark for HOI editing, and a new metric that jointly evaluates the trade-off between successful interaction editing and identity preservation. Extensive experiments show that InteractEdit outperforms 23 existing methods, providing a strong baseline for future HOI editing research.

Method

HOI components are disassembled into subject, object, and background descriptors during inversion. SeRA regularization enables non-rigid edits by capturing essential identity features while suppressing overfitting to structural details. Selective inversion preserves pretrained interaction priors while adapting to the source image's identity. Editing reassembles these components with the target interaction, using SeRA weights delta to guide the diffusion model.

BibTeX

If you use our work in your research, please cite:

@article{HOE2026InteractEdit,
    title = {Reference-free human-object interaction editing},
    journal = {Neurocomputing},
    pages = {134869},
    year = {2026},
    issn = {0925-2312},
    doi = {https://doi.org/10.1016/j.neucom.2026.134869},
    url = {https://www.sciencedirect.com/science/article/pii/S0925231226022678},
    author = {Jiun Tian Hoe and Weipeng Hu and Wei Zhou and Chao Xie and Ziwei Wang and Xudong Jiang and Yap-Peng Tan and Chee Seng Chan},
}
@inproceedings{hoe2026onehoi,
  title={OneHOI: Unifying Human-Object Interaction Generation and Editing},
  author={Hoe, Jiun Tian and Hu, Weipeng and Jiang, Xudong and Tan, Yap-Peng and Chan, Chee Seng},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2026}
}