Dominik Renzel, Ralf Klamma
Advanced Community Information Systems (ACIS) Group
Chair of Computer Science 5 (Databases & Information Systems)
RWTH Aachen University, Germany
Abstract: The current paradigm of global service orientation poses great challenges to traditional Requirements Engineering (RE). Traditional techniques do not scale sufficiently and are widely unaware of community contexts. Consequently, the innovation potential of specialized niche communities often remains untapped by service providers. In Large Scale Social Requirements Engineering (LASSRE), communities should be enabled to express their requirements and trace their realization, while service providers should be supported in discovering relevant innovative requirements to maximize their impact. LASSRE should thereby support an ongoing negotiation process from the spark of an idea to its full realization in a new product or feature between end-users and providers. LASSRE employs social software concepts (e.g. commenting, voting, communication, sharing artifacts) widely known from social networking platforms for community-centered collaboration in RE in a portal for end-user communities. At the same time, LASSRE must integrate this social software portal with a DevOps environment supporting a well-defined software development process. Based on earlier conceptual groundwork, we present Requirements Bazaar, a Web platform for LASSRE, integrated into a state-of-the-art Open Source DevOps environment. We furthermore report evaluation results and experiences from more than one year of productive operation, mainly in ICT research project contexts.
1 INTRODUCTION
Within only a few years, information technology has undergone a radical shift towards global and open service orientation. This shift is challenging traditional Requirements Engineering (RE) for information systems tailored to organizations. The basis of traditional requirements engineering approaches is determined by organizational structure and boundaries. Organization-spanning collaboration and globalization, but also technical developments such as cloud-based service-oriented architectures are fundamentally changing this situation. Services are used across and between enterprises in a geographically distributed and increasingly mobile manner. Basic service functionality is therefore orchestrated to usable solution elements and partial processes in multiple, constantly changing working contexts. Adaptation during system use, but also system evolution over longer time periods are natural consequences in such contexts. However, the decisions for adaptation and evolution are no more only incumbent upon fixed organizational structures, but upon communities of interested and engaged end-users and other stakeholders from multiple organizational or even private contexts. Innovative ideas from end-user communities should be deliberately included into further development. The social aspects of RE, in particular the need to incorporate social context and to emphasize negotiation of stakeholder goals have been highlighted as significant changes in RE for the 21st century [1]. This change was also reflected in agent and goal-oriented requirements modeling techniques [2], [3]. The innovation potential of end-users [4], [5], [6] and Communities of Practice [7] in the Long Tail [8] has long been recognized in research and currently finds its way into practice [9]. Innovations originating from niche communities not seldomly transition into the mainstream later. An example from our own work is a set of multimedia annotation tools, originally created for a researcher community specialized in the documentation of cultural monuments in Afghanistan. In our current research context we find that - with slight modifications and customizations - the same tools appear to be highly valuable for educational purposes in construction, one of the largest industry sectors in Germany. In particular, RE must support the identification and harnessing of the Long Tail [8] innovation potential known to be borne within large numbers of small, specialized, emergent, and sometimes highly innovative Communities of Practice (CoP) [7]. Long Tail CoP requirements are increasingly interesting for service providers as new opportunities to create innovation and impact within and across communities. Failure in identifying these CoPs often renders innovative and relevant requirements inaccessible to service providers. Furthermore, RE must integrate with the aforementioned increasingly global and distributed collaboration practice among service providers [10]. Scaling to a multiplicity of stakeholders from multiple disciplines [11] and managing the uncertainty in mutual understanding of requirements and constraints by both service providers and requesters [12] are essential prerequisites and major challenges to RE at the same time. In particular, RE must overcome the challenge of impaired stakeholder communication [13]. The well-known communication gap between domain specialists (i.e. service requesters) and IT specialists (i.e. service providers) leads to major problems, either during early RE phases or in the worst case to products of minor quality in late development phases. Distributed global service development thus requires a Large-Scale Social Requirements Engineering (LASSRE) platform for bringing together stakeholders from diverse community contexts into an open, traceable and community-aware social process of collaborative requirements elicitation, negotiation, prioritization and realization [14]. Such a platform must effectively support different groups of stakeholders in reaching their specific goals. In particular, service requesters must be enabled and motivated to intuitively state, negotiate and trace the realization of their needs in a community-aware manner. Service providers should receive precise information on requirements prioritization by stakeholders within and across communities in order to support their decision for or against a realization and finally to maximize their impact. Ideally, such a platform should bridge the aforementioned communication gap and integrate into working practices of all stakeholder groups in the most unobtrusive manner to find adoption. Driven by the question of how a software platform can most effectively support LASSRE, we first conducted a survey comparing nine classes of existing RE tools against a set of features and properties essential for SRE. The main finding is that none of the surveyed solutions is completely suited for SRE, thus motivating us to move forward with an own system overcoming the gaps in existing tools. In particular, means for effective requirements prioritization were mostly found to be missing. Furthermore, various strands of work in the literature on end-user participation in innovation and co-creation processes [4], [6], [9] guided the design of such a system. In this paper, we present Requirements Bazaar (http://requirements-bazaar.org), an Open-Source Social Software for traceable requirements elicitation and negotiation with an emphasis on intuitive requirements specification and workspace integration. We elaborate on the design and implementation of Requirements Bazaar into productive service development environments and report on evaluations and experiences made during more than one year of productive operation, mostly in the context of ICT research projects focused on service innovations in various Long Tail communities. Requirements Bazaar is the result of multiple years of research and development [14], [15, [16]. In 2013, Requirements Bazaar won the prestigious best demo audience award at the 21st IEEE International Conference on Requirements Engineering (RE'13) in Rio de Janeiro, Brazil [16]. The rest of this paper is structured as follows. In Section 2 we present results of our survey comparing existing RE systems regarding their suitability for SRE and point to a few upcoming solutions dedicated to LASSRE from the RE research literature. In Section 3 we present Requirements Bazaar and its features in detail. In Section 4 we discuss the outcomes of several evaluations as well as user experiences collected during productive operation in ICT projects. Finally, in Section 5 we draw conclusions and provide an outlook to future work.
2 STATE-OF-THE-ART REQUIREMENTS ENGINEERING TOOLS
Traditional RE expects requirements analysts to actively extract needs from passive stakeholders. In contrast, the vision of community-aware LASSRE depends on active stakeholders who express their needs on their own without the active intervention of RE specialists. In preparation for this work we conducted a survey on existing RE software solutions. In particular, we scanned available publications and/or end-user documentation on each of the surveyed solutions with respect to the following key features and properties required for LASSRE. Target Users (TU) examines whether the solution, in particular with regard to its work flow and usability, is designed for Requirements Analysts (RA), Software Developers (SD), End-Users (EU) or ideally All Stakeholders (ALL). For an effective requirements negotiation process, all stakeholders should be addressed. Open Access (OA) examines if viewing, providing, or working on requirements is restricted to a closed stakeholder group or is ideally open for everyone. Community Lock-in (CL) examines whether the solution inherently restricts to a specific domain or product from the perspective of all stakeholders. In other words, it should be easy for communities to migrate to another solution without too many barriers (e.g. data migration). An ideal solution should not exhibit community lock-in. Elicitation Tools (ET) examines whether stakeholders can develop, evolve and specify requirements collaboratively. Intuitive Specification (IS) examines to which (ideally low) extent technical literacy or training are required for the expression of requirements on stakeholder side. Requirements Management (RM) examines to which (ideally high) extent the solution provides features for documenting, analyzing, tracing and prioritizing requirements. Finally, Adaptive Prioritization (AP) examines to which (ideally high) extent developers can obtain personalizable or community-specific requirements prioritizations. In total we examined nine classes of individual or combined RE solutions from both industry and academia: traditional and agile requirements management tools (e.g. DOORS, TeamPulse, PivotalTracker), forums [17], (semantic) Wikis [18-23], issue trackers [24, 25], tools explicitly motivating end-user participation in RE [26, 27], and informal Open Source Software Development (OSSD) platforms [28, 29]. Lim et al.’s snowballing approach of applying social networks and collaborative filtering for requirements prioritization [30, 31] was in particular inspiring for our work towards using a recommender system approach for requirements prioritization, taking into account rating and social network analysis measures for stakeholder importance. However, all of these approaches are only partially suited for LASSRE. An overview of our results is given in Table 1. A red cell-coloring and a (-) symbolizes a missing feature or a negative property, green (+) an available feature or positive property and yellow (o) anything in between.
Most notably, none of the surveyed solutions exposes all of the above key features and properties. 50% of the solutions target selected stakeholder groups only, thus rendering the solutions unsuitable for SRE in the first place. Fortunately, open access is given for most solutions. For access to issue tracker systems or other developer-related solutions, open access is technically possible, but often restricted to selected groups by policy. Experiences from multiple independent projects showed that non-developers are overwhelmed by the technical concepts and jargon behind issue tracking systems and are thus not willing to use these systems for stating their requirements. Furthermore notably, most solutions lock-in their particular communities in terms of restricting to certain providers or individual services, either technically or by policy. Requirements management is supported by most tools. Tools used for informal requirements elicitation do not support RM, but therefore allow for intuitive requirements specification, which is in particular important in terms of a low entry barrier for end-users. The support for elicitation tools beyond mere text input is also very limited. Here, only end-user RE tools and snowballing offer alternatives, such as image annotations, multimedia storytelling, etc. More than 50% of the solutions do not allow for intuitive requirements specification. Finally, only the snowballing class, mainly represented by [30, 31] allows for advanced requirements prioritization. Given a suitable tool for SRE, participating communities are likely to state a large amount of requirements, resulting in an overburden on developers to review and rank submissions by their importance. Thus, we consider tool support for requirements prioritization as indispensable. Recently, a trend towards dedicated SRE platforms is clearly visible in the RE community [32, 33]. Todoran envisions StakeCloud [33] as cloud resource marketplace with integrated means for communicating requirements for new resources if no fitting resources could be found. A similar use case was discussed in the context of our evaluation, where end-users search for learning widgets in a widget store and can state requirements in case they did not find what they were looking for. Knauss [32] emphasizes the need for adaptiveness to a variety of different end-user contexts and envisions a framework explaining the different end-user context types and their role for requirements elicitation, matching our goal of community-aware SRE. However, since both approaches are in early stages of development, we cannot make any conclusive statements on their suitability to SRE, yet.
3 REQUIREMENTS BAZAAR
In this section we present Requirements Bazaar as an Open-Source Social Software platform for LASSRE. Its primary focus is to bring together communities and service providers into a collaborative requirements elicitation and negotiation process to make the innovation potential of niche communities accessible to service providers for the good of both groups. Communities are supported to intuitively express and trace their requirements and eventually receive a realization. Service providers are supported in discovering relevant innovative requirements to maximize impact with a realization. In this section we provide details on four key features of Requirements Bazaar supporting LASSRE: intuitive requirements specification, a workflow for co-creation, workspace integration and personalizable requirements prioritization. A Intuitive Requirements SpecificationThe first identified challenge is the form of communication between community members coming from service requester and provider backgrounds. While service providers (in particular developers) tend to prefer the provision of formal requirement specifications (cf. [34]), service requesters (end-users in particular) tend to describe their unfulfilled needs by mildly structured or unstructured, easy-to-write texts, sometimes accompanied by pictures and mostly in the context of user stories. As a compromise, the Bazaar specifies community requirements by the aggregation of basic metadata and optional artifacts further detailing the requirement specification (cf. Figure 1). In particular, an artifact can draw upon intuitive media such as annotated images and templated or free-text use case stories, equally practical for users and developers. Furthermore, we allow file uploads and URLs as generic support for additional emergent artifact types. The artifact-based approach seeks to simplify collaborative requirements refinement by allowing every stakeholder to add own artifacts.
 Figure 1. Specification of Community Requirements
Figure 2 shows the user interface of a requirement page served by Requirements Bazaar. The top center area renders metadata and artifacts, organized in a carousel. The right sidebar shows community information for a requirement, in particular the roles of its participants. The left sidebar allows different actions, depending on the user's role (end-user, developer) and the requirement's status (open/assigned/realized; see next section). The lower area allows social operations such as voting, sharing, commenting, contributing artifacts and involvement into collaborative prototype testing, as described in the next section in more detail.
 Figure 2. Example requirement specification collaboratively produced by Requirements Bazaar participants
Another identified challenge is to enable service requesters and providers to innovate together. According to [4, [6], [9], this can be achieved by continuously integrating communities into the entire service design and development process. Accordingly, Requirement Bazaar’s workflow combines innovation and system development in one model. Figure 3 depicts the workflow, divided in five phases. Depending on the phase, the Bazaar allows different co-creation operations: reporting new requirements, refining requirements by adding artifacts or contributing to the discussion, negotiating by voting or commenting, providing/testing a prototype/solution and acknowledging a finished solution. In the initial Idea Generation phase a stakeholder reports an unfulfilled need in the form of an open requirement with initial metadata and artifacts. In the Idea Selection phase the requirement is refined and negotiated by all participating stakeholders and can be selected for realization by one service provider taking the lead. Once a service provider commits to leading to a solution, the requirement is assigned, thus transitioning to the Idea Realization phase. Negotiation and refinement continue. The lead developer invites co-developers and links testable prototypes. Once realized and sufficiently negotiated, the requirement enters the Idea Acknowledgement phase, where the final solution is discussed, possibly leading to new requirements.
 Figure 3. Requirements Bazaar Workflow
The last identified SRE participation challenge is the minimization of entry barriers. For service requesters the platform must not only provide a comfortable requirements reporting and co-creation environment, but also make them aware of its existence in the first place. From a service provider point of view, the platform’s practicability is threatened by a certain coordination overhead caused by the co-creation approach. Consequently, Requirements Bazaar is designed to lower this overhead by integrating with the workspaces of both service requesters and providers (cf. Figure 4). Any stakeholder can follow requirements, thus effectively subscribing to receive event notifications from Requirements Bazaar via email. Furthermore, Requirements Bazaar provides a minimal jQuery plug-in that can be used to instrument arbitrary Web applications with means for integrated reporting of new requirements directly from within the respective application, mostly belonging to the service requester workspace. The improve flag in the top left corner of Requirements Bazaar's user interface is powered by this plug-in, effectively collecting new requirements for Requirements Bazaar itself. Another integration of this plug-in is described in more detail on the example of a Web-based platform for widget-based Responsive Open Learning Environments (ROLE) in the featured tool integration article "Integrating Requirements Bazaar into the ROLE SDK" of this E-letter. On the side of service providers and developers, issue trackers are widely used communication hubs with own commenting and file distribution features. By automatically pushing new comments and artifacts to issue trackers, developers do not have to monitor the Bazaar once the idea realization phase is initiated. Furthermore, changes in issue status are synchronized to the corresponding requirement in the Bazaar, thus providing service requesters with indicators on the level of completion. A more detailed and technical description of this kind of integration is given in the featured tool integration article "Integrating Requirements Bazaar with the Issue Tracker JIRA" of this E-letter.
 Figure 4. Requirements Bazaar Workspace Integration
As discussed in Section 2, effective and personalized means for requirements prioritization are indispensable with respect to scalability. In addition to using aggregated up/downvoting results [35] as relevance criterion, we saw potential in considering additional data sources for receiving improved prioritization quality. For example, we observed that for some participants heavily discussed requirements were more interesting, even if their up/downvotes would tell a different story. Therefore, we equipped Requirements Bazaar with a generic framework for examining the suitability of multiple available analysis data sources for scoring the relevance of requirements. Therefore, we considered multiple facets of data available from regular Requirements Bazaar operation. User Rating Analysis (URA) data is computed from participants' up/downvotes [35] on requirements and is considered the de-facto baseline technique for requirements prioritization. User Monitoring Analysis (UMA) [36] data is computed from information on multiple facets of activity around a requirement (e.g. commenting, contributing artifacts), thus serving as additional indicator for requirement relevance. If available, UMA data on an existing service, for which a requirement was stated, reveals the service’s popularity to a certain extent. For example, realizing a requirement for a popular service used often by many is more likely to generate higher impact for the provider than realizing a requirement for a widely unused service. Social Network Analysis (SNA) [37, 38] applied on the stakeholder networks evolving from the active communication and collaboration on requirements with the help of Requirements Bazaar yields information on the roles and importance of individual stakeholders and reveals community structures. For example, a requirement stated by an influential stakeholder, e.g. a community expert, might be more relevant than a requirement stated by a community novice. However, as [39] suggests, the contrary might also be the case. Given our generic and extensible framework, it would also be possible to apply other techniques such as sentiment analysis for relevance scoring. Formally, the scoring framework is based on a three-level hierarchical linear weighted model constructed from scoring categories, factors and measures. It serves as foundation for computing a single relevance score s(r) for a requirement r, in turn usable for ranking requirements by their relevance (cf. Equation 1).
 wCi represent category weights individually adjustable by each service provider, wFj factor weights defined for each category, wMk measure weights defined for each scoring factor. The sum of all weights per level equals 1. Mk(r) represent the value of a measure Mk for a given requirement r. The model assumes linearity of the relationship between scoring factors and the relevance of a requirement. In addition, it assumes normality of the error distribution. All prerequisites for the hierarchical linear combination of measures, factors and categories to a single score are thus fulfilled. Requirements Bazaar wraps each scoring factor as a weighted sum of scoring measures into separate units called scoring providers. With the additional concept of indirect scoring providers we enable the combination, reuse and nesting of multiple measures, as long as all assumptions for linear regression still apply. To assist the discovery of the most relevant requirements, all scoring providers are assigned to one of the following three categories. User Rating includes all scoring providers whose scores reflect the results of community-based requirements negotiations. User Behavior scoring factors are based on observable user behavior such as activity on the Bazaar or service usage. Community Belonging scoring factors predict for each individual developer whether a requirement is related to (sub-)communities the developer might be interested in. For exploring the relative relevance of scoring factors, all category weights can be adjusted, motivated by the observation that service providers have individual preferences wrt. scoring factor importance. Although technically possible on all levels, weights can only be adjusted via the Requirements Bazaar user interface on the category level. We deliberately wanted to avoid overwhelming end-users with too many configuration options for the different scoring providers. The final goal however is to tune the system for reasonable default weights, ideally fitting most end-users' preferences. Table 2 lists all scoring providers currently active in Requirements Bazaar.
As default weighting, Requirements Bazaar uses a combination of all scoring providers with equal relative factor weights. Internally used scoring measure weights were preset to reasonable initial values. For example, the scoring provider Bazaar activity uses the UMA-based scoring measures Number of recent artifact additions and Number of recent views. As a starting point, we assumed the number of artifact additions to be ten times more expressive for estimating a requirement’s ’hotness’ and set the scoring factor weights accordingly. Figure 5 shows the discovery page of Requirement Bazaar listing requirements ranked by their computed relevance scores. Separate rank orders are provided for requirements in open, assigned and realized stages (cf. Section 3.B). For each requirement, a break-down of influences from individual scoring providers as well as indicators for relevance in different categories are displayed. The left sidebar allows the selection of scoring modes with two presets (Votes only, Balanced) as well as the free configuration of category weights with three sliders. The Balanced preset reflects the default weighting with equal weights for all categories. For the Votes only preset, all weights are set to zero, except for the Social Like scoring provider.
 Figure 5. Requirements discovery incl. means for adapting requirements relevance ranking to presets or personalized configurations
4 EVALUATION & USER EXPERIENCES A Initial Short Term EvaluationWith the availability of a functional prototype of the Requirements Bazaar, we conducted a short-term study to evaluate Requirements Bazaar’s suitability to LASSRE. Nine subjects (N=9) from different, slightly overlapping community contexts situated in the area of Technology-Enhanced Learning (TEL) were divided in two groups representing service requesters and providers. Subjects were instructed to collaboratively carry out the Requirements Bazaar workflow (cf. Section 3.B), using all parts of the system relevant for their group. To anticipate difficulties in coming up with real and innovative requirements on command within short time, subjects were allowed to report both real and fake requirements. After the evaluation session, we administered a questionnaire including 27 5-level Likert-items for assessing the success of Requirements Bazaar with respect to the five dimensions System Quality, Information Quality, User satisfaction, Individual Impact and Community Impact (cf. [40]). One special item asked for the estimated overall success of the Bazaar with regard to suitability as a LASSRE platform. This special item enabled us to perform a multi-variate regression analysis for the assessment of the most important influencing factors of overall success. An additional free-text item explicitly asked for the most popular feature(s). Another special item was a free-text item targeting feature completeness of Requirements Bazaar, asking explicitly which features are missing. Furthermore, we included items for a separate evaluation on the quality of the relevance scoring framework, which are out of the scope of this work. Given an almost equal amount of responses from service requesters and providers, we first analyzed the overall success of Requirements Bazaar. The questionnaire item overall success was rated high on average with 4.375 of 5 points. Among the highest rated factors/features were access convenience (workspace integration) with an average score of 4.625, improvement of user/developer communication (avg. 4.5) and collaboration motivation (avg. 4.125). On the other side, factors related to productivity improvement and personal/community goal contribution were controversial. While the end-user group’s average rating for overall success was >4, the developer group’s average rating of ~3 indicated mediocrity. In later interviews with the participants, we found that this discrepancy is mainly attributable to the artificiality of the laboratory setting and the evaluation tasks. In particular, participants reported that a fast-forward run through the co-creation workflow coupled with fake requirements were problematic. Developers were not able yet to assess the potential benefits the Bazaar offers them, because fake requirements rendered them unable to explore real requirements and to experience workspace integration in real-life practice. For this reason we furthermore evaluated Requirements Bazaar usage in real practice over a longer period of time as part of our project work (cf. next section). Altogether, both groups attested to the Bazaar’s success in enabling communities to participate in LASSRE. Notably, there was a perfect correlation between answers to the overall success item and all factors in the system satisfaction dimension, i.e. the perceived satisfaction with the technical realization. After the exclusion of these items, a multi-variate regression analysis resulted in a model with goodness-of-fit of R2 = 0.774, thus significantly explaining 77.4% of the overall success through the factors community goal contribution, collaboration motivation and timeliness of presented information. Multiple subjects identified the ability to relations between requirements as a missing feature. Finally, we concluded that already at this early stage Requirements Bazaar supported community-aware LASSRE. However, given the criticism about the artificiality of our fast-forward evaluation approach, we decided to publicly launch Requirements Bazaar on the Web and observe the system in real-life productive use. B User Experiences from Long Term Productive UseThe productive deployment of Requirements Bazaar has been continuously used for requirements engineering in the Learning Layers project for the collection of architectural requirements. The architectural requirements collected and refined from the DoW, context cards, user stories, design ideas were split into functional and non-functional requirements. In the end, there were 42 functional requirements and 26 non-functional requirements available. The functional requirements obtained were ingested into Requirements Bazaar to facilitate a collective voting process with the goal to achieve a ranking of the elicited functional requirements as input for a later Quality Function Deployment (QFD) step to develop Houses of Quality (HoQ) 41 for different application domains served by the project. Partners were asked to express their opinion through casting a vote on the most important requirements. Through this collective process, all the existing requirements were rated, enabling the prioritization of requirements. The ranking was constructed by sorting the requirements list according to the scores obtained after the voting procedure. The most popular requirement (i.e. Activity Tracker) obtained a total number of 16 positive votes, while the last requirement in the hierarchy was listed with only 2 votes. Moreover, partners were also encouraged to comment on the requirements, in order to allow further refinement of the available requirements descriptions. The resulting weighted list of requirements served as input for the generation of multiple HoQ. This integration is described in more detail in the featured tool integration article "Turning User Requirements Into Technical Features With the House of Quality" of this special issue. Currently, many of the collected high-priority requirements were picked by project developers, who are now actively developing realizations. For this purpose, requirements were transferred to the project-own installation of Atlassian JIRA with the help of Requirements Bazaar's workspace integration facilities (cf. Section 3.C). This integration is described in more detail in the featured tool integration article "Integrating Requirements Bazaar with the Issue Tracker JIRA" of this E-letter. Furthermore, feedback on system usage was reported as a set of improvement suggestions. In particular the voting process was suggested to become possible and more clearly visible on the Requirements Discovery Page. In the original version this was intentionally avoided to force users into reading the full description of a requirement before allowing them to vote. Furthermore, users suggested to limit the number of active votes per user as one means to avoid the “rich-get-richer” effect found in voting data. Another means was indirect voting by pairwise comparisons of requirements in a gamified fashion, similar to elements of the Analytic Hierarchy Process AHP [42]. Furthermore, community and project awareness should be supported by adding community or project-centered requirements discovery pages and tagging, thus allowing for filtering for those requirements relevant for the respective context. As observations during productive use showed, most end-users were content with the baseline of voting-based requirements relevance scoring. However, some users - mostly on the side of service providers - explicitly reported the usefulness of the more sophisticated weighted relevance scores from multiple scoring provider sources.
5 CONCLUSION
In this paper we presented the design, prototype, implementation as well as initial evaluation results and user experience reports during productive use of Requirements Bazaar, a Web-based social software tool for LASSRE. Based on a literature study and a survey of existing RE tools we derived the need for a communication and collaboration platform, which supports all stakeholders in reaching their particular goals wrt. RE, integrated into their particular working practice: service requesters in stating and negotiating their requirements in an intuitive and traceable manner, optimally leading to a realization; service-providers in getting access to innovative ideas from service requesters across Long Tail CoPs, prioritized by their particular relevance. In a survey on existing state-of-the-art RE tools, we found that none of them was completely suited for LASSRE, thus motivating us to move forward with Requirements Bazaar as a new tool for LASSRE. Requirements Bazaar mainly consists of two parts. The social software part enables collaborative, intuitive and traceable requirements elicitation, negotiation and management, integrated into stakeholder workspaces. An extensible and generic requirements relevance scoring framework enables the access to configurable requirements prioritizations. In a first evaluation we tested the Bazaar with a group of stakeholders from the TEL community and equipped the recommender system with an initial set of relevance scoring providers based on user monitoring, user ratings and stakeholder SNA. User feedback was mostly positive across stakeholder groups, although slightly better on the end-user side. A regression analysis on the results of a post-evaluation survey indicates system quality, community goal contribution, collaboration motivation and timeliness as significant success factors for Requirements Bazaar, thus confirming our intentions for the stakeholder communication part of LASSRE. Although not conclusive, analysis results of the integrated requirements relevance scoring framework showed that the precision of requirement relevance rankings benefits from the inclusion of additional data sources besides pure up/downvoting results. Given its extensibility, the framework still serves as a valuable research tool for further investigations on relevant ranking factors for requirements prioritization. Requirements Bazaar continues its productive use in the Learning Layers project, where many of the collected requirements are currently actively developed. For the future, we envision an even more generic applicability of the Requirements Bazaar to other domains and classes of services. Thus, Requirements Bazaar will be subject to further development, refinement and evaluation. A ready-to-use productive installation of Requirements Bazaar is available at http://requirements-bazaar.org. Information material in the form of videos, posters, references to publications are available on http://requirements-bazaar.org/info. A major part of Requirements Bazaar's code is available as BSD-licensed Open Source Software on SourceForge. Requirements Bazaar is currently undergoing a heavy refactoring and improvement process, based on the feedback we received during more than one year of productive use. The complete re-engineered code will be available on GitHub, again as Open Source Software.
Acknowledgements
The research leading to these results received funding from the European Commission’s 7th Framework Programme (FP7/2007-2013) under grant agreements 231396 (Responsive Open Learning Environments) and 318209 (Learning Layers).
REFERENCES
[1] B. Nuseibeh and S. Easterbrook, “Requirements engineering: a roadmap,” in Proceedings of the Conference on The Future of Software Engineering, ser. ICSE ’00. New York, NY, USA: ACM, 2000, pp. 35–46. [Online]. Available: http://dx.doi.org/10.1145/336512.336523 [2] E. S. K. Yu, “Towards Modelling and Reasoning Support for Early-Phase Requirements Engineering,” in Proceedings of the 3rd IEEE Int. Symp. on Requirements Engineering (RE’97) Jan. 6-8, 1997, Washington D.C., USA,1997, pp. 226–235. [Online]. Available: http://www.cs.toronto.edu/pub/eric/RE97.pdf [3] E. Yu, P. Giorgini, N. Maiden, and J. Mylopoulos, Social Modeling for Requirements Engineering, ser. Cooperative Information Systems. Mit Press, 2011. [4] E. von Hippel, “Lead Users: A Source of Novel Product Concepts,” Management Science, vol. 32, no. 7, pp. 791–805, 1986. [5] G. L. Lilien, P. D. Morrison, K. Searls, M. Sonnack, and E. von Hippel, “Performance Assessment of the Lead User Idea-Generation Process for New Product Development,” Management Science, vol. 48, no. 8, pp. 1042–1059, 2002. [6] H. W. Chesbrough, Open Innovation: The New Imperative for Creating And Profiting from Technology. Harvard Business School Press, 2003. [7] E. Wenger, Community of Practice: Learning, Meaning, and Identity. Cambridge: Cambridge University Press, 1998. [8] C. Anderson, The Long Tail: Why the Future of Business Is Selling Less of More. Hyperion, 2006. [9] K. Diener and F. Piller, The Market for Open Innovation: Increasing the Efficiency and Effectiveness of the Innovation Process. RWTH Aachen University, Technology and Innovation Management Group, 2009. [10] E. S. Raymond, The Cathedral and the Bazaar. Sebastopol, CA, USA: O’Reilly Media, 1999. [11] B. H. C. Cheng and J. M. Atlee, “Research Directions in Requirements Engineering,” in 2007 Future of Software Engineering, ser. FOSE ’07. Washington, DC, USA: IEEE Computer Society, 2007, pp. 285–303. [Online]. Available: http://dx.doi.org/10.1109/FOSE.2007.17 [12] L. Liu, E. Yu, and H. Mei, “Guest Editorial: Special Section on Requirements Engineering for Services: Challenges and Practices,” IEEE Transactions of Service Computing, vol. 2, no. 4, pp. 318–319, 2009. [13] M. F. Costabile, P. Mussio, L. Parasiliti Provenza, and A. Piccinno, “End users as unwitting software developers,” in Proceedings of the 4th International Workshop on End-User Software Engineering, ser. WEUSE ’08. New York, NY, USA: ACM, 2008, pp. 6–10. [Online]. Available: http://dx.doi.org/10.1145/1370847.1370849 [14] E. L.-C. Law, A. Chatterjee, D. Renzel, and R. Klamma, “The Social Requirements Engineering (SRE) Approach to Developing a Large-Scale Personal Learning Environment Infrastructure,” in 21st Century Learning for 21st Century Skills, ser. Lecture Notes in Computer Science, D. Hutchison, T. Kanade, J. Kittler, J. M. Kleinberg, F. Mattern, J. C. Mitchell, M. Naor, O. Nierstrasz, C. Pandu Rangan, B. Steffen, M. Sudan, D. Terzopoulos, D. Tygar, M. Y. Vardi, G. Weikum, A. Ravenscroft, S. Lindstaedt, C. D. Kloos, and D. Hernández-Leo, Eds. Berlin and Heidelberg: Springer, 2012, vol. 7563, pp. 194–207. [15] R. Klamma, M. Jarke, A. Hannemann, and D. Renzel, “Der Bazar der Anforderungen - Open Innovation in emergenten Communities,” Informatik-Spektrum, vol. 34, no. 2, pp. 178–191, 2011. [16] D. Renzel, M. Behrendt, R. Klamma, and M. Jarke, “Requirements Bazaar: Social Requirements Engineering for Community-Driven Innovation,” in 2013 IEEE 21st International Requirements Engineering Conference (RE), Rio de Janeiro, Brazil, July 15-19, 2013, pp. 326–327. [Online]. Available: http://dx.doi.org/10.1109/RE.2013.6636738 [17] P. Laurent and J. Cleland-Huang, “Lessons Learned from Open Source Projects for Facilitating Online Requirements Processes,” in Proceedings of the 15th International Working Conference on Requirements Engineering: Foundation for Software Quality, ser. REFSQ ’09. Berlin, Heidelberg: Springer-Verlag, 2009, pp. 240–255. [Online]. Available: http://dx.doi.org/10.1007/978-3-642-02050-6_21 [18] S. Lohmann, P. H. Heim, and K. Lauenroth, “Web-based Stakeholder Participation in Distributed Requirements Elicitation,” in Proceedings of the 2008 16th IEEE International Requirements Engineering Conference, ser. RE ’08. Washington, DC, USA: IEEE Computer Society, 2008, pp. 323–324. [Online]. Available: http://dx.doi.org/10.1109/RE.2008.16 [19] S. Lohmann, S. Dietzold, P. H. Heim, and N. Heino, “A Web Platform for Social Requirements Engineering,” in Software Engineering 2009: Workshop-Band, Fachtagung des GI-Fachbereichs Softwaretechnik, 02.-06.03.2009 in Kaiserslautern, ser. LNI, J. Münch and P. Liggesmeyer, Eds., vol. P-150. Gesellschaft für Informatik, 2009, pp. 309–315. [20] B. Hoenderboom and P. Liang, “A Survey of Semantic Wikis for Requirements Engineering,” University of Groningen, The Netherlands, Tech. Rep. RUG-SEARCH-09-L03, 2009. [21] F. Adisa, P. Schubert, and F. Sudzina, “The Living Requirements Space: Towards the Collaborative Development of Requirements for Future ERP Systems,” in Scandinavian Information Systems Research, ser. LNBIP, K. Kautz and P. A. Nielsen, Eds., Springer Berlin Heidelberg, 2010, vol. 60, pp. 34–49. [Online]. Available: http://dx.doi.org/10.1007/978-3-642-14874-3_3 [22] D. Wu, D. Yang, and B. Boehm, “Finding Success in Rapid Collaborative Requirements Negotiation Using Wiki and Shaper,” in Proceedings of the 2010 43rd Hawaii International Conference on System Sciences, ser. HICSS ’10. Washington, DC, USA: IEEE Computer Society, 2010, pp. 1–10. [Online]. Available: http://dx.doi.org/10.1109/HICSS.2010.210 [23] C. Solis and N. Ali, “Distributed Requirements Elicitation Using a Spatial Hypertext Wiki,” in Proceedings of the 2010 5th IEEE International Conference on Global Software Engineering, ser. ICGSE ’10. Washington, DC, USA: IEEE Computer Society, 2010, pp. 237–246. [Online]. Available: http://dx.doi.org/10.1109/ICGSE.2010.35 [24] C. R. Prause, M. Scholten, A. Zimmermann, R. Reiners, and M. Eisenhauer, “Managing the Iterative Requirements Process in a Multi-national Project Using an Issue Tracker,” in Proceedings of the 2008 IEEE International Conference on Global Software Engineering, ser. ICGSE ’08. Washington, DC, USA: IEEE Computer Society, 2008, pp. 151–159. [Online]. Available: http://dx.doi.org/10.1109/ICGSE.2008.14 [25] D. Bertram, A. Voida, S. Greenberg, and R. Walker, “Communication, Collaboration, and Bugs: The Social Nature of Issue Tracking in Small, Collocated Teams,” in Proceedings of the 2010 ACM Conference on Computer Supported Cooperative Work, ser. CSCW ’10. New York, NY, USA: ACM, 2010, pp. 291–300. [Online]. Available: http://dx.doi.org/10.1145/1718918.1718972 [26] A. Rashid, “OpenProposal: Towards Collaborative End-User Participation in Requirements Management By Usage of Visual Requirement Specifications,” in Requirements Engineering Conference, 2007. RE ’07. 15th IEEE International, 2007, pp. 371–374. [27] A. Hannemann, C. Hocken, and R. Klamma, “Community Driven Elicitation of Requirements with Entertaining Social Software,” in Software Engineering 2009: Workshop-Band, Fachtagung des GI-Fachbereichs Softwaretechnik, 02.-06.03.2009 in Kaiserslautern, ser. LNI, J. Münch and P. Liggesmeyer, Eds., vol. P-150. Gesellschaft für Informatik, 2009, pp. 317–328. [28] J. Cleland-Huang, H. Dumitru, C. Duan, and C. Castro-Herrera, “Automated support for managing feature requests in open forums,” Commun. ACM, vol. 52, no. 10, pp. 68–74, Oct. 2009. [Online]. Available: http://dx.doi.org/10.1145/1562764.1562784 29] R. Vlas and W. N. Robinson, “A Rule-Based Natural Language Technique for Requirements Discovery and Classification in Open-Source Software Development Projects,” in Proceedings of the 2011 44th Hawaii International Conference on System Sciences, ser. HICSS ’11. Washington, DC, USA: IEEE Computer Society, 2011, pp. 1–10. [Online]. Available: http://dx.doi.org/10.1109/HICSS.2011.28 [30] S. L. Lim, “Social Networks and Collaborative Filtering for Large-Scale Requirements Elicitation,” PhD, University of New South Wales, 2010. [31] S. L. Lim, D. Quercia, and A. Finkelstein, “StakeSource: Harnessing the Power of Crowdsourcing and Social Networks in Stakeholder Analysis,” in Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 2, ser. ICSE ’10. New York, NY, USA: ACM, 2010, pp. 239–242. [Online]. Available: http://dx.doi.org/10.1145/1810295.1810340 [32] A. Knauss, “On the Usage of Context for Requirements Elicitation: End-User Involvement in IT Ecosystems,” in 2012 20th IEEE International Requirements Engineering Conference (RE), Chicago, IL, USA, September 24-28, 2012, M. P. E. Heimdahl and P. Sawyer, Eds. IEEE, 2012, pp. 345–348. [33] I. Todoran, “StakeCloud: Stakeholder Requirements Communication and Resource Identification in the Cloud,” in 2012 20th IEEE International Requirements Engineering Conference (RE), Chicago, IL, USA, September 24-28, 2012, M. P. E. Heimdahl and P. Sawyer, Eds., IEEE, 2012, pp. 353–356. [34] IEEE Computer Society, New York, NY, USA, IEEE Standard 830-1998 - IEEE Recommended Practice for Software Requirements, 1998. 35] D. Zhang, R. Mao, H. Li, and J. Mao, “How to Count Thumb-Ups and Thumb-Downs?: An Information Retrieval Approach to User-Rating based Ranking of Items,” in Proceedings of the 34th international ACM SIGIR Conference on Research and Development in Information Retrieval, ser. SIGIR ’11. New York, NY, USA: ACM, 2011, pp. 1223–1224. [Online]. Available: http://dx.doi.org/10.1145/2009916.2010130 [36] A. Croll and S. Power, Complete Web Monitoring. Sebastopol, CA, USA: O’Reilly, 2009. [37] S. Wasserman and K. Faust, Social Network Analysis: Methods and Applications. Cambridge University Press, 1994. [38] A. Clauset, M. E. J. Newman, and C. Moore, “Finding community structure in very large networks,” Phys. Rev., vol. E 70, no. 6, 2004. [39] A. Niknafs and D. M. Berry, “The Impact of Domain Knowledge on the Effectiveness of Requirements Idea Generation during Requirements Elicitation,” in 2012 20th IEEE International Requirements Engineering Conference (RE), Chicago, IL, USA, September 24-28, 2012, M. P. E. Heimdahl and P. Sawyer, Eds. IEEE, 2012, pp. 181–190. [40] W. H. DeLone and E. R. McLean, “Information Systems Success: The Quest for the Dependent Variable,” Information Systems Research, vol. 3, no. 1, pp. 60–95, 1992. [41] [10] J. R. Hauser and D. Clausing, “The House of Quality,” Harvard Business Review, vol. 66, no. 3, pp. 63-73, 1988. [42] T. L. Saaty, Decision Making for Leaders - The Analytic Hierarchy Process for Decisions in a Complex World, Pittsburgh, PA, USA: RWS Publishing, 1995.
 |
E-Letter > STCSN E-Letter Vol.2 No.3 >