Publications

You can also find my articles on my Google Scholar profile.

Journal Articles


Joint Discovery of Object and Action Symbols through Effect Prediction for Robotic Manipulation Planning

Published in , 2026

To perform complex manipulation planning, autonomous robots are required to abstract continuous, high-dimensional sensorimotor interactions into discrete object and action representations. Earlier work either categorized objects based on visual appearances, which fails to distinguish objects that appear similar but behave differently, or based on effects under interaction, but was limited to predefined actions. To address these limitations, we propose a model that jointly discovers high-level manipulation primitives and object categories through a binary bottleneck layer, trained to predict multi-modal outcomes, including object motion, contact, and force feedback, from random interaction data. Building on these discovered binary representations, we leverage a discrete planning method that uses intermediate steps in the predicted effect trajectory to enable partial action executions for precise low-level control. Additionally, we evaluate our framework’s generalization capabilities on novel objects by assigning object categories through comparing a small number of interaction effects with the predicted effects of learned object symbols, enabling few-shot generalization based on behavior rather than visual similarity. We conduct experiments on tabletop repositioning and stacking tasks, and confirm that our effect-driven planning approach outperforms both a state-of-the-art method and a visual-based alternative in planning precision across seen and novel objects.

Recommended citation: B. Kilic, B. Kartal, F. Dogangun, E. Oztop, E. Ugur, "Joint Discovery of Object and Action Symbols through Effect Prediction for Robotic Manipulation Planning", 2026.
Download Paper

Bilevel Planning with Learned Symbolic Abstractions from Interaction Data

Published in , 2026

Intelligent agents must reason over both continuous dynamics and discrete representations to generate effective plans in complex environments. Previous studies have shown that symbolic abstractions can emerge from neural effect predictors trained with a robot’s unsupervised exploration. However, these methods rely on deterministic symbolic domains, lack mechanisms to verify the generated symbolic plans, and operate only at the abstract level, often failing to capture the continuous dynamics of the environment. To overcome these limitations, we propose a bilevel neuro-symbolic framework in which learned probabilistic symbolic rules generate candidate plans rapidly at the high level, and learned continuous effect models verify these plans and perform forward search when necessary at the low level. Our experiments on multi-object manipulation tasks demonstrate that the proposed bilevel method outperforms symbolic-only approaches, reliably identifying failing plans through verification, and achieves planning performance statistically comparable to continuous forward search while resolving most problems via efficient symbolic reasoning.

Recommended citation: Dogangun, F., Kilic, B., Bahar, S., & Ugur, E. (2026). Bilevel Planning with Learned Symbolic Abstractions from Interaction Data. arXiv preprint arXiv:2603.08599.
Download Paper

Conference Papers


Predictability-Based Curiosity-Guided Action Symbol Discovery

Published in , 2025

Discovering symbolic representations for skills is essential for abstract reasoning and efficient planning in robotics. Previous neuro-symbolic robotic studies mostly focused on discovering perceptual symbolic categories given a pre-defined action repertoire and generating plans with given action symbols. A truly developmental robotic system, on the other hand, should be able to discover all the abstractions required for the planning system with minimal human intervention. In this study, we propose a novel system that is designed to discover symbolic action primitives along with perceptual symbols autonomously. Our system is based on an encoder-decoder structure that takes object and action information as input and predicts the generated effect. To efficiently explore the vast continuous action parameter space, we introduce a Curiosity-Based exploration module that selects the most informative actions—the ones that maximize the entropy in the predicted effect distribution. The discovered symbolic action primitives are then used to make plans using a symbolic tree search strategy in single- and double-object manipulation tasks. We compare our model with two baselines that use different exploration strategies in different experiments. The results show that our approach can learn a diverse set of symbolic action primitives, which are effective for generating plans in order to achieve given manipulation goals.

Recommended citation: B. Kilic, A. Ahmetoglu and E. Ugur, "Predictability-Based Curiosity-Guided Action Symbol Discovery" 2025 IEEE International Conference on Development and Learning (ICDL), 2025.
Download Paper

Learning Action Primitives in Manipulation Robots

Published in , 2025

Intelligent agents sense and act in a continuous space at the sensorimotor level. However, most of our daily tasks require manipulating a finite number of objects using a fixed set of skills, suggesting the use of perceptual and action primitives. Previous studies either learn perceptual primitives from a set of high-level actions or learn these actions from hand-coded states. In this study, we propose a neural architecture that allows simultaneous learning of action primitives together with object-level perceptual primitives from interaction data. The interaction process is guided by an active learning module that enables the search for effect-focused action parameters. The experiment results show that our model can find meaningful action primitives.

Recommended citation: B. Kilic, A. Ahmetoglu and E. Ugur, "Learning Action Primitives in Manipulation Robots," 2025 33rd Signal Processing and Communications Applications Conference (SIU), Sile, Istanbul, Turkiye, 2025, pp. 1-4, doi: 10.1109/SIU66497.2025.11111831.
Download Paper