Whenever someone asks me what my research project is about, I usually open by saying we’re calling it collaborative knowledge discovery from data. That’s a nice, convenient way of putting it in a nutshell, but it immediately calls for some elaboration, especially on the meaning of the term “collaborative”. Technically, any activity that involves two or more people working together toward a common goal is collaborative, but this definition doesn’t get us very far, because in knowledge discovery you typically have at least someone who knows about the technology and someone who knows about the application domain. It’s not unheard of for one person to know about both, but still, I think it’s safe to say that collaboration is the rule rather than the exception here.

To narrow it down a bit, the kind of collaboration we’re talking about is remote and synchronous. In other words, the participants are not located in the same place, but they can all simultaneously edit whatever it is they’re collaborating on and see the effects of each other’s edits in real time. This implies that there must be some kind of online environment where the collaboration takes place; think something like Google Docs or MS Office Online, only for KDD artifacts such as datasets, algorithms and processing pipelines.

Even this is not a particularly novel idea in itself, as there are collaboration platforms already available where you can develop just these sorts of things. Therefore in KDD-CHASER we’re upping the ante even further by focusing specifically on collaborative knowledge discovery from personal data, driven by the data owner who cannot be assumed to have any particular technology or domain expertise. It’s a bit of a niche, which of course makes our eventual results somewhat less generalisable, but it also makes it considerably easier to spot opportunities for novel research contributions.

To me, the most interesting problems here are not necessarily associated with knowledge discovery as such but with the things that need to happen before knowledge discovery can take place. After all, from the data owner’s perspective the point of collaborating with experts is basically to have the actual discovery done by people who are better equipped for it in terms of skills and tools. This doesn’t mean, however, that the data owner’s role in the collaboration is limited to being a passive data source; on the contrary, it is the data owner’s needs that drive the entire process of collaborative KDD in the special case we’re considering.

The first problem that a data owner may encounter on the way to a successful collaboration is that they don’t even know anyone they could collaborate with, so the first thing the collaboration platform should do is provide a matchmaking service that brings together people who have data with people who have the right sort of expertise to help turn it into something more valuable. After the matchmaking follows the really interesting part: negotiation. What kind of knowledge is the data owner interested in? What is actually achievable, given the available data and the extent to which the data owner is willing to share it with the expert? What is the expert expecting to get in compensation for their efforts? The collaborators need to find the answers to such questions among themselves, and the collaboration platform should support them in this.

The bare minimum is to provide the collaborators with some kind of communication channel, but this is something that would be required anyway, and it’s hardly a research problem from a computing standpoint. However, there’s a lot more to negotiation than just talking, and I’m interested to see what I might do to help things along in this area. Privacy, for example, is traditionally close to my heart and something that I want to address also here, because one of the things to be determined through negotiation is how much of their data the data owner is prepared to trust their collaborators with, considering that the latter may be the KDD equivalent of someone they just matched with on Tinder.

It’s been pretty clear from the start that whatever we manage to accomplish in my current project, it’s not going to be a comprehensive solution to all the problems of collaborative KDD, even within the niche we’ve carved for ourselves. What we can realistically shoot for, though, is a model that shows us what the collaboration process looks like and gives us an understanding of where the major problems are. The software I’m building will basically be a collection of candidate solutions to a select few of these problems, and it will hopefully be something I can continue to build on when my MSCA fellowship is over.