Abstract— Tactile sensing has gained interest among robotics researchers due to its utility in obtaining geometric information about unknown physical contact. This can be used for contact-rich dexterous manipulation tasks when interacting with the environment. To acquire multimodal contact information in a compact structure, we have developed a new sensor called ViTacTip. This sensor physically fuses tactile and vision information in a single sensor and can also be used as a proximity sensor and force sensor after calibration. ViTacTip features a transparent skin which enables the see-through mechanism to capture fine features of objects during contact. The embedded biomimetic tips on ViTacTip can amplify touch motions. The obtained tactile images offer a natural fusion of visual and tactile information within one device, where the texture feature can be obtained and the background provides the object’s appearance information. We also developed a ViTac sensor (without biomimetic tips) and a TacTip sensor (without a transparent skin for the see-through mechanism) for comparative studies. We evaluated the proposed sensors based on four tasks: i) proximity estimation, ii) grating identification, iii) pose regression, and iv) contact localization and force estimation. In the grating identification task, ViTacTip achieves 99.72%, which is slightly better than TacTip’s 94.60%. It performs the best in both pose and force estimation tasks with the minimum error
of 0.08mm and 0.03N, compared to ViTac’s 0.12mm and 0.15N. Results indicate that ViTacTip outperforms the other sensors in all the aforementioned tasks.