mark: A photo of Mark kneeling on top of the Taal Volcano in the Philippines. It was a long hike. (Default)
Mark Smith ([staff profile] mark) wrote2010-05-09 11:42 am

protected content security

There's a bunch of talk going around right now about the whole issue of content security, and trusting the people who host your content and have access to it. I wanted to talk on that for a moment as it's something that is really important to me.

The only people that are authorized to view protected content on Dreamwidth that they don't otherwise have access to are [staff profile] denise and myself. We are the only people with the proper access level. On top of that, it's not automatic -- in order to view the protected content, every time one of us visits a URL we have to edit it and add "viewall=1" to the end of it. It's a very manual process (for good reason). It's also logged -- and I don't know about Denise, but I review the logs regularly, just like every other security log we have.

The second level of access is for people who have access to the production servers that run Dreamwidth. When someone has the ability to log in to our servers, they have full access to the data on the databases and could in theory access protected content. The only people with server access are again myself and Denise, plus our two sysadmins: [personal profile] matthew, who used to work for LJ (before and during Six Apart), and [personal profile] alierak, who I've known for a decade and I trust completely.

That's it. The four of us.

At some point, it comes down to trust. We need the ability to work on the servers, so there are always going to be a set of people who have the ability to see private data. This isn't something that we can feasibly get rid of, either. The data exists on the servers (that's how we can show it to you and the people who are authorized to see it) and we need access to those servers to maintain them. The data isn't just sitting around visible to us, though -- it's tucked away in the database and requires a lot of manual effort to dig out, unzip, and connect to a user account. We never see post content accidentally.

In the end, I think that the best that I can offer anybody is to be explicit about who has access (and what kind of access they have) and to personally watch the security logs. I watch to make sure we don't have unauthorized access to our servers, and I look for unauthorized access to private data as well. It's part of the routine, and it's something I take very seriously. Having dealt with some problems related to this in the past (on other projects, with other people) it's not something I want to see Dreamwidth have to go through.

I'm happy to talk about this, if anybody has any thoughts, comments, or questions.
damned_colonial: Convicts in Sydney, being spoken to by a guard/soldier (Default)

[personal profile] damned_colonial 2010-05-09 07:07 pm (UTC)(link)
We were talking about something vaguely similar the other week at work, and we decided to trigger an email whenever someone did something with their elevated privs, so that it would be more visible to those who needed to know about it and the person doing it would be accountable.

Thinking along those lines, could you send a notification to the user, as well as logging, when someone with privs views a protected entry? The text of the message could include something like, "This is most likely in relation to a support request you raised" or whatever is necessary to explain why it's happening and reassure the user. It feels, to me, a bit like the email you get that says "Someone, possibly you, has requested a password change."
Edited 2010-05-09 19:08 (UTC)
eagle: Me at the Adobe in Yachats, Oregon (Default)

[personal profile] eagle 2010-05-09 08:35 pm (UTC)(link)
One other thing along those lines to worry about is that you could end up in a situation where you're legally not permitted to notify the user that their content is being viewed (a valid search warrant, for instance). While this is exactly the case where I suspect most users would like to be notified, since you're a US business, you simply can't. So saying that you'll notify users when the content is viewed is to some extent promising something that you can't reliably deliver.

My purely personal perspective, admittedly as someone who knows a lot about how this sort of thing works since I'm a professional sysadmin, is that the site users have to trust you and Denise by necessity and, beyond that, I'm not sure there's a lot of utility in trying to enable community audit of your activities. I'm also not sure that there's much utility in notification if there's no option or decision available to the user.

One thing you could consider is, in a future world in which you may want to grant more fine-grained access to additional people, would be to distinguish between access under the user's control and access that is done by a site admin. For instance, when submitting a support ticket, a user could potentially get a checkbox saying "allow senior support people access to my restricted content" which they could check if the request seemed to warrant it. If they chose not to check it, then their support request may take longer if it requires such access until you or Denise or similar staff had a chance to look at it.

Not sure if the gain is worth the complexity, though.
noxie: friendly girl smiling (Default)

[personal profile] noxie 2010-05-10 08:11 am (UTC)(link)
For instance, when submitting a support ticket, a user could potentially get a checkbox saying "allow senior support people access to my restricted content" which they could check if the request seemed to warrant it.

I would love that.
pne: A picture of a plush toy, halfway between a duck and a platypus, with a green body and a yellow bill and feet. (Default)

[personal profile] pne 2010-05-10 10:52 am (UTC)(link)
One other thing along those lines to worry about is that you could end up in a situation where you're legally not permitted to notify the user that their content is being viewed (a valid search warrant, for instance). While this is exactly the case where I suspect most users would like to be notified, since you're a US business, you simply can't.

That reminds me of's warrant canary (and others like it).
aveleh: Close up picture of a vibrantly coloured lime (Default)

[personal profile] aveleh 2010-05-09 08:43 pm (UTC)(link)
If it's automatic, I suspect people would use it as a form of harassment; sending in a report purely so that the poster would get a notification that it was looked at. That's not a reason to avoid doing this, but I'm not sure that the benefits are worth it.

An alternative could be something like you guys have talked about it terms of sharing TOS investigations; doing it the other way? "In May, x journals were looked at for a, b, and c reasons". Let people know how rare or common it is?
msilverstar: (Default)

[personal profile] msilverstar 2010-05-09 11:44 pm (UTC)(link)
I like this, especially if it's auto-generated by the database records. While it's self-reporting, it's enough to give everyone some feeling for the process, and change over time (so give percentages too).
Edited (where "this" = periodic reports of what accesses have been done) 2010-05-09 23:46 (UTC)
phi: (Default)

[personal profile] phi 2010-05-11 03:21 pm (UTC)(link)
I like this idea as well.
rydra_wong: Lee Miller photo showing two women wearing metal fire masks in England during WWII. (Default)

[personal profile] rydra_wong 2010-05-09 08:53 pm (UTC)(link)
The viewall utility is used for many uses, not all of which I would want to send emails for. (Reports of a credible suicide threat, investigating a ToS violation, me verifying imports/deletes/purges/things, etc etc...)

Maybe it would be useful to have a FAQ section or something about this, spelling out under what circumstances viewall gets used, who can use it, and emphasizing what DW does to protect privacy and ensure that the ability to access private/locked posts won't be abused.
noxie: friendly girl smiling (Default)

[personal profile] noxie 2010-05-10 08:10 am (UTC)(link)
+ 1

I think this would be a great thing for the FAQs.
cesy: Home is where the <3 is (Dreamwidth) (Dreamwidth)

[personal profile] cesy 2010-05-09 07:09 pm (UTC)(link)
You rock. Thank you for stating this clearly, and spelling out the details.
seryn: flowers (Default)

[personal profile] seryn 2010-05-09 07:31 pm (UTC)(link)
I think it's terrific that you were willing to explain this to us.

Personally I try not to have locked content that is going to be damaging if it gets out because I heard the rumors about [other places] sending restricted-access content to advertisers for keyword mining.
sporky_rat: One of the Awesome Future Dudes from Bill and Ted's Excellent Adventure. (big man around here)

[personal profile] sporky_rat 2010-05-09 07:33 pm (UTC)(link)
I want to say that I really appreciate the transparency that you have been supporting and are willing to tell us just who has access to our private data. I'm also grateful that only these few have access.

I think I'm trying to have a grateful party right now, but I'm lacking the words to properly express it. Really, truly, thank you.
poulpette: Stick-figure of a smiling head, raising heart pompoms. Bottom half says YAY!!! (Misc - devs = awesome)

[personal profile] poulpette 2010-05-09 07:36 pm (UTC)(link)
See, that's part of the reason why I trust you and Denise. You're up front with us. If an issue comes to your attention, and needs to be addressed by top level management, you guys address it publicly. And, even better, you do so clearly, and with as much details as possible/reasonable.

Thank you for the explanation.
Edited (fixed spelling, akward sentence.) 2010-05-09 19:56 (UTC)
sharpest_asp: Nate Ford sitting on a bench, Sophie Devereaux resting against his lap (Default)

[personal profile] sharpest_asp 2010-05-09 07:53 pm (UTC)(link)
Thank you for posting this, but you guys already have my complete trust.
princessofgeeks: (Default)

[personal profile] princessofgeeks 2010-05-09 08:11 pm (UTC)(link)
thank you.
ursamajor: people on the beach watching the ocean (Default)

[personal profile] ursamajor 2010-05-09 08:30 pm (UTC)(link)
This kind of practice is what makes Dreamwidth feel like the safest interactive place for me to call "home" on the internet these days. Thank you.

PGP Option?

[personal profile] krellis 2010-05-09 08:30 pm (UTC)(link)
One interesting feature, if enough people really want more assurance about this, would be to add a "PGP posting" feature to the software itself - it'd be a pseudo-security-level, really just the same as a normal "locked" type of entry, but you would be able to select from some set of PGP public keys that you've uploaded for the content to be encrypted with, and then, using some out of band means, provide the private key half of that PGP key to anyone you want to be able to read it. I can't think of a secure way of Dreamwidth doing the decryption half without it again being possible for you guys to read the content, but there are browser plugins designed for doing PGP work on webmail systems that would work.

It's quite possibly excessive, and not something I personally would necessarily use or care much about, just an idea that popped into my head if you wanted to offer something for those who are really paranoid or really have something they feel strongly that they need to keep confidential.
msilverstar: (viggo 09)

Re: PGP Option?

[personal profile] msilverstar 2010-05-09 11:55 pm (UTC)(link)
I think an option for gentle encryption would be nice. Not designed to stop the NSA, but OK for dealing with non-hackers.

As long as it's clearly explained that this is no guarantee of privacy, it would be another useful tool.
eagle: Me at the Adobe in Yachats, Oregon (Default)

Re: PGP Option?

[personal profile] eagle 2010-05-10 01:28 am (UTC)(link)
There's something to be said for simple symmetric-key encryption (using AES for instance) even if Dreamwidth also stores the key and decrypts on the fly for authorized users. It means that the data is encrypted at rest and no one can see it by accident.

A minor gain probably not worth the complexity, but that sort of server-side encryption is becoming increasingly common in the storage world. (Although there the concern is often about backups, which are frequently sent off-site to unaffiliated storage companies who shouldn't be able to read the stuff they're storing.)
pseudomonas: (Default)

Re: PGP Option?

[personal profile] pseudomonas 2010-05-10 06:21 pm (UTC)(link)
Hmm, interesting. I like the idea of users being able to broadcast stuff securely like this. *but* I think if we do something like this it's got to be really rigorous. It's fine with stuff that's access-locked being by-and-large good enough, but things that are claiming to be crypto have to be done right - giving a false sense of security is much worse than just not offering the service.

If the encryption is done browser-side with another plugin then Dreamwidth never touches the plaintext or the keys and the only thing that needs security auditing is the plugin. All DW has to do is provide a nice way of tagging the text. Or am I missing something?
lanterne_rouee: i believe in dreamwidth plus a typewriter (dw believe typewriter)

[personal profile] lanterne_rouee 2010-05-09 08:53 pm (UTC)(link)
Thanks for making a post like this.
ilyena_sylph: picture of Labyrinth!faerie with 'careful, i bite' as text (Default)

[personal profile] ilyena_sylph 2010-05-09 09:56 pm (UTC)(link)
Thanks for this post. It's a good thing to know.
syderia: cyber wolf (geek)

[personal profile] syderia 2010-05-10 05:23 am (UTC)(link)
Thank you for this post.
blnchflr: Faniversity - DW campus (Faniversity)

[personal profile] blnchflr 2010-05-10 06:36 am (UTC)(link)
Thank you for taking the time to make this post - I actually assumed more DW people in theory had access to users' locked content, so I feel extra reassured.
noxie: friendly girl smiling (Default)

[personal profile] noxie 2010-05-10 08:16 am (UTC)(link)
Mark, thank you so much for addressing this issue publicly! I had emailed you, asking about it, and I can't tell you how happy it makes me to see you address this so promptly. That is awesome, and it really makes me trust you guys more than I already did. It's very reassuring to know how seriously you take this, and that (at least for now) only 4 people have access to locked content.

Again, thank you so much for this post!
birggitt: Argentina dreamsheep (Argentina)

[personal profile] birggitt 2010-05-10 10:48 am (UTC)(link)
Thank you so much for clarify this issue. I am one of those persons who like private stuff... well private *laughs* And to know who could be looking at my locked entries and why, and to know you are actually monitoring for unauthorized access makes me feel better about the whole thing.
Again, thanks!

[personal profile] miss_haitch 2010-05-10 11:51 am (UTC)(link)
Here via [personal profile] sofiaviolet -- just wanted to second what others have said and thank you for clarifying with this post.
alierak: (Default)

[personal profile] alierak 2010-05-10 04:54 pm (UTC)(link)
I would also point out that server access gives us the theoretical ability to read network traffic (even https) off the wire, not just database content, so there is possibly more at stake there than just protected entries. I can't immediately think of a circumstance in which I would be looking at that type of content in order to do my job, unless I already had reason to believe that the traffic or database entries constituted an attack on Dreamwidth's server resources. I do not work support requests unless specifically asked, and will not access private information to do so. I do try to work on difficult-to-diagnose bugs, but thanks to [personal profile] sophie, it's simple enough to troubleshoot things on a Dreamhack instead.
egret: Capt. Janeway reading a paid (Default)

[personal profile] egret 2010-05-11 12:54 am (UTC)(link)
Thank you for running the journaling system with fairness and honesty!
oona: (sad)

[personal profile] oona 2010-06-30 12:34 pm (UTC)(link)
Off topic, but: I'm sorry to hear about your dog having to undergo surgery. I know how that is. I hope he will be well and fully recovered soon. Pets are family. God bless.

[personal profile] ephemeralsprite 2010-09-05 05:56 pm (UTC)(link)
And yet another comment, just in case, to repeat that my formerly posted questions are answered in the wiki.
I'm so not used to being able to edit my comments
amai_kaminari: minekura beer, icon by amai-kaminari (Default)

[personal profile] amai_kaminari 2010-09-18 02:28 am (UTC)(link)
Thank you for clarifying your stance on the matter. As a recent LJ ex-pat, I am really pleased just to know that DW's owners are willing to have a dialogue about the needs of its customers.

Thank you for that.