Give a researcher a test and they can use it for a study. Give them a testing software and they can…

Sharing competence assessment items endangers the psychometric validity. As they are released to the broader public, study participants might learn about them. This “burns” the instrument, as the effect of knowing a particular item interferes with the measurement of the domain skill of a person.

On the other hand, sharing a software used to administer the items can hardly influence participants, since it does not include the actual items. On the other hand there is a number of advantages associated in free sharing:

  • Networking: Shared software can become a point of contact for scientisits with similar interests, opening up venues for sharing knowledge and exploring research opportunities.
  • Transparency: Sharing is documenting - when others can run the software used in an assessment they can reproduce and reason about the experience of participants.
  • Paying forward: Starting with technology based assessment from scratch is hard. When you share your groundwork, new researchers in the field can build upon it to jumpstart their own assessments.
  • Collaboration: By sharing software and code, others can contribute to it, from finding bugs, improving written documentation, to adding new features. The original authors do stay in control about their “brand” of the software though. Using public repositories like Github, they have the last word about all proposed changes and additions.

Open source

“Open source” is the keyword for freely sharing software. Its rote requirement is, that you do publically share a software and its source code, and licence it in a way that allows others to freely use and modify it. Such a licence usually includes a clause to indemnify you as far as legally possible.

There are many established open source licenses out there. A common one is the MIT-licence:

Copyright (c) <year> <copyright holders>

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the “Software”), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

Copyleft

“Copyleft” licenses include a clause requiring all derivates of the work to be put under the same licence.

Such a clause complicates how confidential materials, like test items, can be integrated into the software in a way that does not require their public release. For that reason that class of licenses seems like a bad choice for psychometrical instruments relying on participants not knowing the items beforehand.

Licenses like MIT that do not include such a copyleft-clause allow derivates to be re-licensed freely, so there is no requirement to share modified software or source code.

Further reading (German):