.NET core should not SPY on users by default

[ghost]

@blackdwarf @piotrMSFT I am very disappointed to discover that .NET core comes with a hidden and enabled spy utility that reports on its users. (Lakshanf/issue2066/telemetry dotnet/cli#2145). Apparently, MS has learned nothing from the backclash against Windows 10 spying on users. I suspect many will not want to install .NET core for this reason, which is a shame because .NET core is otherwise cool.

[richlander]

Our recent blog post discusses the addition of telemetry to the .NET Core tools. See: https://blogs.msdn.microsoft.com/dotnet/2016/05/16/announcing-net-core-rc2/#telemetry

Me and the folks on my team are motivated to provide a great product. As you can also see from the blog post, we've made some pretty dramatic changes in RC2. We believe that they are the right ones, but we need both feedback and usage data in order to help us find all of the rough edges. Usage data tends to be more objective in the aggregate and user feedback more insightful, so we do a better job when we have both available.

The data we collect does not identify individual users. We're only interested in aggregate data that we can use to identify trends. The telemetry feature is configurable, so you can turn it on/off at any time. It is also scoped, only applying to tools usage, not the rest of the product. We think that this is a good trade-off and recognize that not everyone will like it. We do know, however, that many people will like the product improvements that will come from this insight.

We intend to share the data. The presence of it will do a lot to define the scope of data. It will also give the community access to the same insight we have. We very much feel that improving .NET Core is a shared need and task. As an example, we would welcome a PR from the community that added another telemetry data point given a strong improvement reason and no loss in anonymity.

We are separately considering opt-in runtime telemetry to learn more about crashes, GC pauses and startup time. There is no way we can get enough insight about the product without that kind of information. We are very focussed on constant improvement and will transparently do what it takes to ensure the product is compelling and competitive.

As an aside, it's been a busy week with shipping RC2 and answering questions. I haven't actually looked at this data yet and I'm one of the primary consumers. I'll be doing that today or tomorrow. I'm looking forward to sharing my insights.

[guardrex]

@richlander

we need both feedback and usage data

Does the telemetry still include arguments provided to the dotnet command? In server hosting scenarios, some may have sensitive arguments passed to the command (for portable apps) that they wouldn't want leaked to MS.

and will transparently do what it takes

The program is not "transparent" IMO.

[terrajobst]

Does the telemetry still include arguments provided to the dotnet command?

My understanding is that this was recently discussed with our privacy team and we concluded that collecting the arguments themselves (hashed or not) is not acceptable per our privacy policies. Not sure whether the code already reflects that, but it's being worked on.

[terrajobst]

The program is not "transparent" IMO.

What would you accept as sufficiently transparent? Not trying to say that we already are sufficiently transparent; I'm trying to understand your concern and what we could do make it better. The product is worked on by various teams who all contribute to the same open source code base on GitHub. Clearly you think that's not sufficient, so I'd like to understand what process would address that.

[guardrex]

@terrajobst Ah! Thanks. I'm glad the arguments are safe on the server.

WRT transparency: There is no indication at the time of install that the dotnet cli is automatically opted-into data sharing. There's no checkbox that will set the opt-out env var. There's no note or link to the GH issue or a Docs page that describes the program and how to opt-out. The privacy policy merely links to the generic MS privacy policy, where there is no mention of the program.

You really have to have heard about this through the GH issues or via chat at JabbR or Slack ... or Wireshark your server I guess. In my mind, that hardly constitutes "transparency."

IMO there is a great risk here for negative PR if the mainstream media gets a hold of this issue that will not be good. It's only a matter of time before some enterprising journalist looking for a scoop picks up on this. The headlines here are not good: "Microsoft caught with sneaky program to spy on companies" ... I know ... I know ... barely accurate given what the data is, how it's shared, and it's use by the teams. You know that doesn't matter one bit when you're trying to sell a newspaper. I was a college newspaper editor. Trust me ... it will not be good if the current disclosures about this program hold to RTM.

[richlander]

@guardrex This is good feedback. We do have a bit more to do to make sure that everything to with telemetry is obvious. We'll make sure that gets into the next release.

GuardRex is exactly right about the lack of transparency and danger you are in for a shitstorm, so it is a good idea to include a checkbox in the installer to make it visible!

Also, you should keep in mind the problem is both privacy AND security. As for security, I think that MS forget that a power user/developer may have hundreds of pieces of software installed. If all these pieces of software (in a stealthy way) report usage back to various servers on the internet, then the security attack surface becomes so large that it is impossible to secure the computer. Hence, many companies will ban your software (especially on Linux servers).

This particular "feature" may undo all the good things that MS is doing with .NET core. Even if I might personally be persuaded to risk my computer and privacy, some of my customers won't. Hence, I will be reluctant to base my development on .NET core because of the customer reaction to spying.

[vcsjones]

I would probably agree that people will be a little miffed by this. Homebrew for OS X recently went through this even though they were well intentioned, did it anonymously, and provided a way to opt out.

I think simply asking people on first use if they'd like to submit telemetry is a good start.

Consider what Yeoman does on first use:

I think people are generally happy to give feedback when asked.

[mmc41]

@blackdwarf
@piotrpMSFT
@richlander
In related news, VS2015 just got into big trouble because spy code was discovered:
https://www.reddit.com/r/cpp/comments/4ibauu/visual_studio_adding_telemetry_function_calls_to/d30dmvu

You should consider learning from such mistakes!

[vcsjones]

Looks like issue dotnet/cli#3404 is tracking implementing notification of telemetry.

[h3smith]

@richlander - as someone looking to deploy projects built with this is healthcare and classified environments, this creates significant challenges. An environment variable is a decent starting point, but build time and local options should also be given to ensure that this data is not collected. I appreciate the desire of you guys, but it introduces security concerns.

[akoeplinger]

@guardrex https://blogs.msdn.microsoft.com/dotnet/2016/06/27/announcing-net-core-1-0/#user-content-net-core-tools-telemetry shows the data points that are collected and the following statement which should make it pretty clear that telemetry only applies to the tools/CLI (i.e. dotnet):

The feature will not collect any personal data, such as usernames or emails. It will not scan your code and not extract any project-level data that can be considered sensitive, such as name, repo or author (if you set those in your project.json). We want to know how the tools are used, not what you are using the tools to build. If you find sensitive data being collected, that’s a bug. Please file an issue and it will be fixed.

[guardrex]

only applies to the tools/CLI (i.e. dotnet)

If you mean it only applies to executing a portable app using dotnet (dotnet .\myapp.dll) and not a self-contained app using corehost (myapp.exe) ... I don't think the language states that clearly. One has to know that you don't consider corehost to be a "tool," and that's not an assumption that I would make.

There is an on-going problem in assuming too much prior knowledge in communication with people (outside of the ASP.NET docs, where a major effort has been made to address this problem). I think writing docs with greater attention to explicit and comprehensive explanations, as annoying and time-consuming as that may be, clears up a great deal of confusion.

Setting this minor confusion aside, I greatly appreciate the effort that has been made to inform everyone about the telemetry program. I still wish that production servers weren't automatically opted-into the program, mostly because (just like @blackdwarf commented in a recent video interview I saw) I hate having to set and maintain env vars on servers ... a total PITA IMO.

[dlebedynskyi]

Guys, this issue is really important. A lot of projects ask for telemetry and it is ok. In fact for a bunch of those like yo, bower and so on dev like me willingly opt in.
But not asking using if he even want and referring to some elua that really no one will read smells. It is horrible negative PR.
Make option to opt in for use. Explain in details what you are going to collect and what not. Do not do it by default.
Otherwise we really will have to block feature or not use dotnet entirely. I really don't think that paranoid security team will even allow devs to deploy this now.

[kspeakman]

Discovering this telemetry has put the plans I had in using .NET Core back on the drawing board. You are essentially refusing to accept an arms-length relationship by including telemetry. Data leakage is a risk even if it isn't user specific. It also creates attack opportunities since attackers now have this plentiful and predictable avenue of communication to go after. Not to mention that once marketing gets wind (if they didn't help drive it in the first place), the data collection will be expanded. Save yourself work by creating more admin/security work for your users (to opt out or block telemetry). Just because it's an industry trend doesn't mean its a good thing to do. </3