hold ball
kick ball
throw ball
hold ball
kick ball
TL;DR: We enables zero-shot human-object interaction edit
Previously available as a preprint titled “InteractEdit: Zero-Shot Editing of Human-Object Interactions in Images”.
Orange indicates target interaction.
dribble ball
hold ball
kick ball
throw ball
hold ball
kick ball
hold skateboard
sit on skateboard
jump skateboard
ride skateboard
jump skateboard
ride skateboard
jump skateboard
ride skateboard
Orange indicates target interaction.
This paper presents InteractEdit, a novel framework for reference-free Human-Object Interaction (HOI) editing that tackles the challenging task of transforming an existing interaction in an image into a new, desired interaction while preserving the identities of the subject and object. Unlike prior image editing tasks such as attribute manipulation, object replacement or style transfer, HOI editing involves complex spatial, contextual, and relational dependencies inherent in HOI. Existing methods often overfit to the source image structure, limiting adaptability to the substantial structural modifications demanded by the new interactions. To address this, InteractEdit disassembles each scene into subject, object, and background components to disentangle intricate HOI relationship, and introduces a selective inversion strategy combined with Selective-Rank Adaptation (SeRA) to leverage pretrained interaction priors while learning visual identity from the source image. This enables a balanced trade-off between interaction editing and identity preservation. We also introduce IEBench, a new benchmark for HOI editing, and a new metric that jointly evaluates the trade-off between successful interaction editing and identity preservation. Extensive experiments show that InteractEdit outperforms 23 existing methods, providing a strong baseline for future HOI editing research.
HOI components are disassembled into subject, object, and background descriptors during inversion. SeRA regularization enables non-rigid edits by capturing essential identity features while suppressing overfitting to structural details. Selective inversion preserves pretrained interaction priors while adapting to the source image's identity. Editing reassembles these components with the target interaction, using SeRA weights delta to guide the diffusion model.
If you use our work in your research, please cite:
@article{HOE2026InteractEdit,
title = {Reference-free human-object interaction editing},
journal = {Neurocomputing},
pages = {134869},
year = {2026},
issn = {0925-2312},
doi = {https://doi.org/10.1016/j.neucom.2026.134869},
url = {https://www.sciencedirect.com/science/article/pii/S0925231226022678},
author = {Jiun Tian Hoe and Weipeng Hu and Wei Zhou and Chao Xie and Ziwei Wang and Xudong Jiang and Yap-Peng Tan and Chee Seng Chan},
}
@inproceedings{hoe2026onehoi,
title={OneHOI: Unifying Human-Object Interaction Generation and Editing},
author={Hoe, Jiun Tian and Hu, Weipeng and Jiang, Xudong and Tan, Yap-Peng and Chan, Chee Seng},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
year={2026}
}