View article

Improving a technology is greatly eased by the existence of a reference architecture paired with a proper evaluation framework in order to have a coherent vision of the components we have to leverage on and to precisely measure and track performances over time. Unfortunately, the situation is very fragmented in the case of keyword-based access to relational data, since both a fully-fledged reference architecture and an evaluation methodology are still missing [2]. In this paper, we start to investigate the problem of the reproducibility of the experimental results for keyword-based access systems. Reproducibility is becoming a primary concern in many areas of science and it is a key both for fully understanding state-of-the-art solutions and for being able to compare against and improve over them [6, 7]. However, there is a lack of commonly shared open source platforms implementing state-of-the-art algorithms for keyword-based access to relational data as, for example, Terrier1 is in the information retrieval field; on the contrary, you mostly need to re-implement each system from scratch [4]. Therefore, as part of a student project during the course on databases of the master degree in computer science at the University of Padua, we considered several state-of-the-art algorithms and we implemented them from scratch, in order to understand the difficulties and pitfalls in implementing them and to go towards a shared implementation of them. The paper is organized as follows: Section 2 presents the related work; Section 3 introduces the implementation of the different algorithm; Section 4 reports the lessons learned; finally Section 5 wraps up the …