Using machine learning for image captioning

Abstract

Modern information systems for e-commerce increasingly integrate product recommendation engines to enhance the user experience. This paper analyzes the application of image captioning techniques based on machine learning, with the goal of generating better recommendations for e-commerce platforms focused on selling clothing. Special attention is given to the limitations of traditional approaches that rely solely on visual similarity between products. Image captions generated by eight different models were evaluated, with large language models performing the best. Subsequently, the proposed solution, which combines the traditional visual similarity approach with semantic similarity analysis of the generated descriptions, was
evaluated across previously defined problems. The results demonstrate the effectiveness and justification of this new approach. Finally, the paper outlines potential directions for future research.