Evergreens

Mark Smith's Journal

Work related musings of a geek.

protected content security

[staff profile] mark
There's a bunch of talk going around right now about the whole issue of content security, and trusting the people who host your content and have access to it. I wanted to talk on that for a moment as it's something that is really important to me.

The only people that are authorized to view protected content on Dreamwidth that they don't otherwise have access to are [staff profile] denise and myself. We are the only people with the proper access level. On top of that, it's not automatic -- in order to view the protected content, every time one of us visits a URL we have to edit it and add "viewall=1" to the end of it. It's a very manual process (for good reason). It's also logged -- and I don't know about Denise, but I review the logs regularly, just like every other security log we have.

The second level of access is for people who have access to the production servers that run Dreamwidth. When someone has the ability to log in to our servers, they have full access to the data on the databases and could in theory access protected content. The only people with server access are again myself and Denise, plus our two sysadmins: [personal profile] matthew, who used to work for LJ (before and during Six Apart), and [personal profile] alierak, who I've known for a decade and I trust completely.

That's it. The four of us.

At some point, it comes down to trust. We need the ability to work on the servers, so there are always going to be a set of people who have the ability to see private data. This isn't something that we can feasibly get rid of, either. The data exists on the servers (that's how we can show it to you and the people who are authorized to see it) and we need access to those servers to maintain them. The data isn't just sitting around visible to us, though -- it's tucked away in the database and requires a lot of manual effort to dig out, unzip, and connect to a user account. We never see post content accidentally.

In the end, I think that the best that I can offer anybody is to be explicit about who has access (and what kind of access they have) and to personally watch the security logs. I watch to make sure we don't have unauthorized access to our servers, and I look for unauthorized access to private data as well. It's part of the routine, and it's something I take very seriously. Having dealt with some problems related to this in the past (on other projects, with other people) it's not something I want to see Dreamwidth have to go through.

I'm happy to talk about this, if anybody has any thoughts, comments, or questions.
09.05.2010 07:07 pm (UTC)

(no subject)

damned_colonial: Convicts in Sydney, being spoken to by a guard/soldier (Default)
Posted by [personal profile] damned_colonial
We were talking about something vaguely similar the other week at work, and we decided to trigger an email whenever someone did something with their elevated privs, so that it would be more visible to those who needed to know about it and the person doing it would be accountable.

Thinking along those lines, could you send a notification to the user, as well as logging, when someone with privs views a protected entry? The text of the message could include something like, "This is most likely in relation to a support request you raised" or whatever is necessary to explain why it's happening and reassure the user. It feels, to me, a bit like the email you get that says "Someone, possibly you, has requested a password change."
Edited 09.05.2010 07:08 pm (UTC)
09.05.2010 08:35 pm (UTC)

(no subject)

eagle: Me at the Adobe in Yachats, Oregon (Default)
Posted by [personal profile] eagle
One other thing along those lines to worry about is that you could end up in a situation where you're legally not permitted to notify the user that their content is being viewed (a valid search warrant, for instance). While this is exactly the case where I suspect most users would like to be notified, since you're a US business, you simply can't. So saying that you'll notify users when the content is viewed is to some extent promising something that you can't reliably deliver.

My purely personal perspective, admittedly as someone who knows a lot about how this sort of thing works since I'm a professional sysadmin, is that the site users have to trust you and Denise by necessity and, beyond that, I'm not sure there's a lot of utility in trying to enable community audit of your activities. I'm also not sure that there's much utility in notification if there's no option or decision available to the user.

One thing you could consider is, in a future world in which you may want to grant more fine-grained access to additional people, would be to distinguish between access under the user's control and access that is done by a site admin. For instance, when submitting a support ticket, a user could potentially get a checkbox saying "allow senior support people access to my restricted content" which they could check if the request seemed to warrant it. If they chose not to check it, then their support request may take longer if it requires such access until you or Denise or similar staff had a chance to look at it.

Not sure if the gain is worth the complexity, though.
10.05.2010 08:11 am (UTC)

(no subject)

noxie: friendly girl smiling (Default)
Posted by [personal profile] noxie
For instance, when submitting a support ticket, a user could potentially get a checkbox saying "allow senior support people access to my restricted content" which they could check if the request seemed to warrant it.

I would love that.
10.05.2010 10:52 am (UTC)

(no subject)

pne: A picture of a plush toy, halfway between a duck and a platypus, with a green body and a yellow bill and feet. (Default)
Posted by [personal profile] pne
One other thing along those lines to worry about is that you could end up in a situation where you're legally not permitted to notify the user that their content is being viewed (a valid search warrant, for instance). While this is exactly the case where I suspect most users would like to be notified, since you're a US business, you simply can't.

That reminds me of rsync.net's warrant canary (and others like it).
09.05.2010 08:43 pm (UTC)

(no subject)

aveleh: Close up picture of a vibrantly coloured lime (Default)
Posted by [personal profile] aveleh
If it's automatic, I suspect people would use it as a form of harassment; sending in a report purely so that the poster would get a notification that it was looked at. That's not a reason to avoid doing this, but I'm not sure that the benefits are worth it.

An alternative could be something like you guys have talked about it terms of sharing TOS investigations; doing it the other way? "In May, x journals were looked at for a, b, and c reasons". Let people know how rare or common it is?
09.05.2010 11:44 pm (UTC)

(no subject)

msilverstar: (Default)
Posted by [personal profile] msilverstar
I like this, especially if it's auto-generated by the database records. While it's self-reporting, it's enough to give everyone some feeling for the process, and change over time (so give percentages too).
Edited (where "this" = periodic reports of what accesses have been done) 09.05.2010 11:46 pm (UTC)
11.05.2010 03:21 pm (UTC)

(no subject)

phi: (Default)
Posted by [personal profile] phi
I like this idea as well.
09.05.2010 08:53 pm (UTC)

(no subject)

rydra_wong: Lee Miller photo showing two women wearing metal fire masks in England during WWII. (Default)
Posted by [personal profile] rydra_wong
The viewall utility is used for many uses, not all of which I would want to send emails for. (Reports of a credible suicide threat, investigating a ToS violation, me verifying imports/deletes/purges/things, etc etc...)

Maybe it would be useful to have a FAQ section or something about this, spelling out under what circumstances viewall gets used, who can use it, and emphasizing what DW does to protect privacy and ensure that the ability to access private/locked posts won't be abused.
10.05.2010 08:10 am (UTC)

(no subject)

noxie: friendly girl smiling (Default)
Posted by [personal profile] noxie
+ 1

I think this would be a great thing for the FAQs.
09.05.2010 07:09 pm (UTC)

(no subject)

cesy: Home is where the <3 is (Dreamwidth) (Dreamwidth)
Posted by [personal profile] cesy
You rock. Thank you for stating this clearly, and spelling out the details.
09.05.2010 07:31 pm (UTC)

(no subject)

seryn: flowers (Default)
Posted by [personal profile] seryn
I think it's terrific that you were willing to explain this to us.

Personally I try not to have locked content that is going to be damaging if it gets out because I heard the rumors about [other places] sending restricted-access content to advertisers for keyword mining.
09.05.2010 07:33 pm (UTC)

(no subject)

sporky_rat: One of the Awesome Future Dudes from Bill and Ted's Excellent Adventure. (big man around here)
Posted by [personal profile] sporky_rat
I want to say that I really appreciate the transparency that you have been supporting and are willing to tell us just who has access to our private data. I'm also grateful that only these few have access.

I think I'm trying to have a grateful party right now, but I'm lacking the words to properly express it. Really, truly, thank you.
09.05.2010 07:36 pm (UTC)

(no subject)

poulpette: Stick-figure of a smiling head, raising heart pompoms. Bottom half says YAY!!! (Misc - devs = awesome)
Posted by [personal profile] poulpette
See, that's part of the reason why I trust you and Denise. You're up front with us. If an issue comes to your attention, and needs to be addressed by top level management, you guys address it publicly. And, even better, you do so clearly, and with as much details as possible/reasonable.

Thank you for the explanation.
Edited (fixed spelling, akward sentence.) 09.05.2010 07:56 pm (UTC)
09.05.2010 07:53 pm (UTC)

(no subject)

sharpest_asp: Nate Ford sitting on a bench, Sophie Devereaux resting against his lap (Default)
Posted by [personal profile] sharpest_asp
Thank you for posting this, but you guys already have my complete trust.
09.05.2010 08:11 pm (UTC)

(no subject)

princessofgeeks: (Default)
Posted by [personal profile] princessofgeeks
thank you.
09.05.2010 08:30 pm (UTC)

(no subject)

ursamajor: people on the beach watching the ocean (Default)
Posted by [personal profile] ursamajor
This kind of practice is what makes Dreamwidth feel like the safest interactive place for me to call "home" on the internet these days. Thank you.
09.05.2010 08:30 pm (UTC)

PGP Option?

Posted by [personal profile] krellis
One interesting feature, if enough people really want more assurance about this, would be to add a "PGP posting" feature to the software itself - it'd be a pseudo-security-level, really just the same as a normal "locked" type of entry, but you would be able to select from some set of PGP public keys that you've uploaded for the content to be encrypted with, and then, using some out of band means, provide the private key half of that PGP key to anyone you want to be able to read it. I can't think of a secure way of Dreamwidth doing the decryption half without it again being possible for you guys to read the content, but there are browser plugins designed for doing PGP work on webmail systems that would work.

It's quite possibly excessive, and not something I personally would necessarily use or care much about, just an idea that popped into my head if you wanted to offer something for those who are really paranoid or really have something they feel strongly that they need to keep confidential.
09.05.2010 11:55 pm (UTC)

Re: PGP Option?

msilverstar: (viggo 09)
Posted by [personal profile] msilverstar
I think an option for gentle encryption would be nice. Not designed to stop the NSA, but OK for dealing with non-hackers.

As long as it's clearly explained that this is no guarantee of privacy, it would be another useful tool.
10.05.2010 01:28 am (UTC)

Re: PGP Option?

eagle: Me at the Adobe in Yachats, Oregon (Default)
Posted by [personal profile] eagle
There's something to be said for simple symmetric-key encryption (using AES for instance) even if Dreamwidth also stores the key and decrypts on the fly for authorized users. It means that the data is encrypted at rest and no one can see it by accident.

A minor gain probably not worth the complexity, but that sort of server-side encryption is becoming increasingly common in the storage world. (Although there the concern is often about backups, which are frequently sent off-site to unaffiliated storage companies who shouldn't be able to read the stuff they're storing.)
10.05.2010 06:21 pm (UTC)

Re: PGP Option?

pseudomonas: (Default)
Posted by [personal profile] pseudomonas
Hmm, interesting. I like the idea of users being able to broadcast stuff securely like this. *but* I think if we do something like this it's got to be really rigorous. It's fine with stuff that's access-locked being by-and-large good enough, but things that are claiming to be crypto have to be done right - giving a false sense of security is much worse than just not offering the service.

If the encryption is done browser-side with another plugin then Dreamwidth never touches the plaintext or the keys and the only thing that needs security auditing is the plugin. All DW has to do is provide a nice way of tagging the text. Or am I missing something?
09.05.2010 08:53 pm (UTC)

(no subject)

lanterne_rouee: i believe in dreamwidth plus a typewriter (dw believe typewriter)
Posted by [personal profile] lanterne_rouee
Thanks for making a post like this.
09.05.2010 09:56 pm (UTC)

(no subject)

ilyena_sylph: picture of Labyrinth!faerie with 'careful, i bite' as text (Default)
Posted by [personal profile] ilyena_sylph
Thanks for this post. It's a good thing to know.
10.05.2010 05:23 am (UTC)

(no subject)

syderia: cyber wolf (geek)
Posted by [personal profile] syderia
Thank you for this post.
10.05.2010 06:36 am (UTC)

(no subject)

blnchflr: Faniversity - DW campus (Faniversity)
Posted by [personal profile] blnchflr
Thank you for taking the time to make this post - I actually assumed more DW people in theory had access to users' locked content, so I feel extra reassured.
10.05.2010 08:16 am (UTC)

(no subject)

noxie: friendly girl smiling (Default)
Posted by [personal profile] noxie
Mark, thank you so much for addressing this issue publicly! I had emailed you, asking about it, and I can't tell you how happy it makes me to see you address this so promptly. That is awesome, and it really makes me trust you guys more than I already did. It's very reassuring to know how seriously you take this, and that (at least for now) only 4 people have access to locked content.

Again, thank you so much for this post!
10.05.2010 10:48 am (UTC)

(no subject)

birggitt: Argentina dreamsheep (Argentina)
Posted by [personal profile] birggitt
Thank you so much for clarify this issue. I am one of those persons who like private stuff... well private *laughs* And to know who could be looking at my locked entries and why, and to know you are actually monitoring for unauthorized access makes me feel better about the whole thing.
Again, thanks!
10.05.2010 11:51 am (UTC)

(no subject)

Posted by [personal profile] miss_haitch
Here via [personal profile] sofiaviolet -- just wanted to second what others have said and thank you for clarifying with this post.
10.05.2010 04:54 pm (UTC)

(no subject)

alierak: (Default)
Posted by [personal profile] alierak
I would also point out that server access gives us the theoretical ability to read network traffic (even https) off the wire, not just database content, so there is possibly more at stake there than just protected entries. I can't immediately think of a circumstance in which I would be looking at that type of content in order to do my job, unless I already had reason to believe that the traffic or database entries constituted an attack on Dreamwidth's server resources. I do not work support requests unless specifically asked, and will not access private information to do so. I do try to work on difficult-to-diagnose bugs, but thanks to [personal profile] sophie, it's simple enough to troubleshoot things on a Dreamhack instead.
11.05.2010 12:54 am (UTC)

(no subject)

egret: egret in Harlem Meer (Default)
Posted by [personal profile] egret
Thank you for running the journaling system with fairness and honesty!
30.06.2010 12:34 pm (UTC)

(no subject)

oona: (sad)
Posted by [personal profile] oona
Off topic, but: I'm sorry to hear about your dog having to undergo surgery. I know how that is. I hope he will be well and fully recovered soon. Pets are family. God bless.
05.09.2010 05:56 pm (UTC)

(no subject)

Posted by [personal profile] ephemeralsprite
And yet another comment, just in case, to repeat that my formerly posted questions are answered in the wiki.
I'm so not used to being able to edit my comments
18.09.2010 02:28 am (UTC)

(no subject)

amai_kaminari: minekura beer, icon by amai-kaminari (Default)
Posted by [personal profile] amai_kaminari
Thank you for clarifying your stance on the matter. As a recent LJ ex-pat, I am really pleased just to know that DW's owners are willing to have a dialogue about the needs of its customers.

Thank you for that.