mark | Recent Entries

If you are in or near or can get to Sydney, NSW, AU this Friday (the 15th), I would love to meet up with you! I am staying near the Circular Quay in the downtown Sydney area.

If you're interested, please comment! If you have any suggestions on places for a (suspected to be) small group to get dinner and hang out for a while, I'd welcome that, too.

Thanks!

Happy Monday -- well, almost, depending on where you are. Who has what for this week's update? What I have on my list:

* Still quiet for the holidays
* Update on me in Sydney and location for get together-ish, also Wellington
* Holiday promotion ending soon
* ...

I know I had more stuff, but being here for Christmas has emptied my mind of most work related topics.

Anybody feel like going through the comments to this post:

http://dw-biz.dreamwidth.org/2794.html

I want a summary of features requested, features demanded, and how many people want them. If ten users say "I absolutely must have pink dingos" then that should be noted. Basically, a priority list of what people want, sorted by number of people and the feature itself.

If you want to take this on, just leave a comment saying so! I'm hoping to have the data in the next week or so. Thanks. :)

...

PS, and also, Merry Christmas! I hope all of you have a wonderful day, whether you celebrate this particular holiday or not.

PPS, over the past 2 months, Dreamwidth has averaged 5,611 posts-per-day. Just going based on the number of documents being indexed by our search system. We're up over 7.4 million posts being indexed right now. (Not all of these are public, of course! But we index most posts.)

Does anybody have anything for the news post?

I'm planning on keeping it fairly short, wishing everybody Happy Holiday of Choice, and advising that we'll be quiet for the next few weeks. No pushes, few patches, many folks are busy... The usual!

If you're looking for me over the next two weeks, I may be sparse. I will be on a plane for most of the 20th and the 28th, and in between I will be at my parents' house doing Christmas. As soon as I get back, we're preparing for New Year's here, and a second Christmas at home.

I will be checking email every day and will be online as much as possible. I will continue to do the weekly news posts, too. But if you're expecting much in the way of patch review or code or fast responses out of me, please keep this in mind. Thanks!

What do we have for the Monday update? So far I have:

* Holiday promotion
* Development update (code tour, bug tour?)

Short list on my side. Whatchagot?

Today has been slow, I've been doing system updates on the production stack. Taking machines out gracefully so nothing misses them and the site doesn't melt down, upgrading them, rebooting them, verifying them, and putting them back in ... very exciting.

I still have to do the more exciting ones. Search will have to go offline for 10 minutes because we don't have two machines running that service. I also can't do the primary load balancer or the master database without taking a downtime, so I'll have to schedule that. But most of the other things I can get done pretty quickly and painlessly.

We're not running anything so close to the edge that I'm worried about system updates messing up our performance. I also did the updates on one test webserver last night and let it run for a while: nobody complained of any weirdness, so nothing untoward seems to have happened in the upgrades.

I'm going to start asking people to upgrade their development environments to Karmic, too, and see how that does. I'm in no hurry to update the production environment, but since we're not running LTS I want to try to make sure we don't end up on archaic unsupported security-hole-ridden versions of things...

Okay, sb-web03 has come back, time to go continue the parade.

I just decommissioned the last two production slices we had active on Slicehost. The original load balancer slice and the admin slice.

They served us well for the ~6 months we used them. There were a number of hiccups along the way (hence our moving) but all in all it wasn't terrible. Reasonable for a beta service.

Just figured I'd note this down here.

I'm priming the pump, this entry is tagged "nnwm09" which means that you will be able to find it in the "find latest posts tagged nnwm09" page:

http://www.dreamwidth.org/latest?tag=nnwm09

Note that this doesn't work with all tags -- only with ones we explicitly allow. For now, that's nnwm09. Even though this entry has nothing to do with nnwm09. ;-)

This is a new feature we're currently working on and testing. It's going through an accelerated roll out because NaNoWriMo starts in a few days. Please let me know if you see any bumps or bruises along the way.

Weekly call. If you have anything for Monday, shout.

The next big feature Denise and I are focusing on is with the update page (draft/scheduled posts). Anyway, I did some stats on popular entry properties (logprops) which are those things that you can change on your entry. Mood, location, music, etc.

mysql> select lpl.name, round(count(*)/2726795*100,4) as 'pct'
            from logprop2 lp, dw_global.logproplist lpl
            where lp.propid = lpl.propid group by 1 order by 2 desc;
+---------------------------------+---------+
| name                            | pct     |
+---------------------------------+---------+
| opt_backdated                   | 94.3205 | 
| import_source                   | 93.6952 | 
| commentalter                    | 61.6888 | 
| picture_keyword                 | 50.8513 | 
| current_moodid                  | 47.2920 | 
| taglist                         | 42.5932 | 
| current_music                   | 22.9734 | 
| current_mood                    | 15.8868 | 
| opt_preformatted                |  7.0147 | 
| used_rte                        |  5.9983 | 
| current_location                |  5.8955 | 
| revnum                          |  5.2326 | 
| revtime                         |  5.2326 | 
| xpost                           |  2.7042 | 
| pingback                        |  1.4610 | 
| opt_screening                   |  0.9731 | 
| opt_nocomments                  |  0.8044 | 
| hasscreened                     |  0.6988 | 
| adult_content                   |  0.4056 | 
| useragent                       |  0.3082 | 
| opt_noemail                     |  0.1677 | 
| sms_msgid                       |  0.0780 | 
| qotdid                          |  0.0635 | 
| adult_content_reason            |  0.0579 | 
| copyright                       |  0.0010 | 
| adult_content_maintainer_reason |  0.0001 | 
| adult_content_maintainer        |  0.0001 | 
| syn_id                          |  0.0000 | 
| syn_link                        |  0.0000 | 
+---------------------------------+---------+
29 rows in set (18.98 sec)

Some of these are straight up imports from LJ (qotdid, copyright?) and don't apply to us. If you filter out the useful ones, you get a really clear picture of which options are often used. Note that this is from the Alpha Centauri cluster only, but I expect it's representative of the entire userbase since we randomly distribute users.

To break this down by feature:

* Set userpic: 51%
* Current mood (id): 48%
* Crossposted: 43%**
* One or more tags: 42%
* Current music: 22%
* Current mood (text): 15%
* Preformatting: 7%
* Current location: 5%
* Comment screening: 1%
* Comments disabled: 0.8%
* Adult content set: 0.4%
* Comments no email: 0.16%
* Adult content reason: 0.05%

That should cover all of the widgets down below. I think this means we could pretty easily split up the options into groups:

* High usage: Userpic, Mood, Tags, Crosspost**, Music
* Low usage: Preformatting, Location, Comment Options, Adult Content

** The Crosspost values differ from the SQL output to the final data. This is because xpost can ONLY be set on entries that were NOT imported. The SQL query pulls data across all posts: including imports. If you factor out imported posts, 43% of all posts on this cluster have been crossposted, hence, the feature is far more popular than suggested by the query itself.

If you have anything for the weekly update, please comment. I'm going to be mentioning site search being live as well as

juliet's book being out.

Also:

If you are on IRC or somewhere and you have an issue that either Denise or I can help with (that's most of them) and you say "hey Denise, are you around", you're shooting yourself in the foot a little because I won't know to answer. I've seen this happen many times. Someone asks for Denise, and she's not around, and hours go by... when I could have helped them out in seconds.

Denise and I have worked hard to make sure that both of us are fairly interchangeable. She's been coding, and I've been handling business and support things as needed. Chances are, most things you might think "oh I need Mark for this" or "oh I need Denise for this" are things either of us can help with.

Thank you!

That time of the week again (woah). Who has something for Monday's update?

I was thinking of calling out a community or two, or interesting people who could use some more readers? Does anybody have any suggestions? (I'll contact them before putting them in the post, of course.)

Starting next week, I'm going to be working full time on Dreamwidth. Most of you know that already. I am going to be experimenting with different types of time management, though, to make sure that I don't end up spending the whole day on IRC and never getting anything done.

I'm going to aim to make these true statements, although they might not be at first or always:

* If you contact me asynchronously (i.e., through email or Bugzilla), I will get back to you within 24 hours. This will be the totally preferred way of contacting me, since it lets me set aside blocks of time to do correspondence and task switch to that mode.

* If you want to talk to me directly, IRC or IM ('xb95' on AIM) are your best bets, but I am going to be disabling these during prime hacking hours. (Which are going to vary from day to day.)

* I am going to schedule "office hours" which will be times that I promise to be 100% available for anything that you want to do synchronously. If you want me to walk you through a patch or just want to hang out and be social, this will be the time. I'll probably use a specific IRC channel for this, but welcome contact via GTalk, AIM, etc. I will set aside several blocks of time each week for this, with varying hours, to try to accomodate all timezones.

* Code reviews should never sit for more than 72 hours. (This allows for weekends.) Except, of course, vacations and other pre-announced periods. And really huge patches that are just going to take a while to review, but I will comment on the bug and say that.

These are what I want to happen. A lot of it comes down to everybody else, though. If you see things languishing, or you have an email that has gone unresponded, kick me. "Hey, Mark, any word?" I'll get the hint and get on it.

Suggestions and comments and thoughts on the above are welcome!

I'm doing the next few updates. Who has stuff for this week's?

Another testing post, please ignore. Still to be ignoring!

Then this post will show up on my test server in less than five seconds, complements of PubSubHubbub.

Edit: It worked. :-)

Okay, it's been some months (4 actually) since Open Beta hit. And the excitement that I felt then really hasn't gone away. Looking around Dreamwidth now, seeing that people are actually using the site we've built, seeing how active

dw_suggestions is, etc etc...

Yeah, I'm pretty stoked. This is an awesome thing we're doing, and it wouldn't be possible without all of the amazing people who have spent so much time working together to build it.

That's it. Warm fuzzies, and bedtime now.

For tonight's adventure, I wrote a Perl script that enables us to easily simulate mulit-master replication in MySQL over secure SSH tunnels.

It almost hurts my brain how much of a hack this is. But it's probably the fastest (least painful) way of getting the two data centers synced up so that when we do the site move this weekend we can pretty quickly get the databases involved to have the same worldview.

In other news... still getting things prepped for the big move. It's a lot of work. We're almost ready, though; the only things that are left are things that really have to wait until closer to the last minute (i.e., setting up cron jobs and workers).

I need to write a playbook too, so that when things actually start rolling I can just check off steps and not have to remember things as we go.

This is a test -- if this works, it should show up almost instantly within FriendFeed. (And anybody else who uses the Google PubSubHubbub hub.)

For some details on what I'm talking about, you can read the technical details here:

http://code.google.com/p/pubsubhubbub/

There was a good video of

brad and Brett Slatkin (the two guys behind it) giving a demo, but I'm not sure where that went. I'll try to dig that up before next week's news post.

Profile

Mark Smith

Dreamwidth

February 2013

S	M	T	W	T	F	S
					1	2
3	4	5	6	7	8	9
10	11	12	13	14	15	16
17	18	19	20	21	22	23
24	25	26	27	28

Syndicate

Page Summary

Active Entries

1: A day in the life...
2: The origin of the name Dreamwidth
3: Portland, OSCON, etc
4: Portland!
5: YAPC::NA 2012 so far!
6: At YAPC::NA 2012!
7: on the previous subject...
8: The meritocracy is harmful.
9: most visited accounts
10: the state of Dreamwidth: load, capacity, etc

Style Credit

Base style: Ciel by carisma_sensei
Theme: Hot Chocolate and Marshmallows by momijizukamori

Expand Cut Tags

No cut tags

Page generated Jun. 20th, 2013 12:06 am

Mark Smith's Journal

Work related musings of a geek.

Recent Entries

meetup in Sydney, Friday?

(no subject)

data gathering

news post tonight

Holiday travel

stuff for Monday

system updates

RIP Slicehost servers

Pump priming.

stuff for Monday?

stats on popular logprops

weekly update, and a note

stuff for Monday update

"office hours"

stuff for Monday

TEST TEST

If this works...

verdict: still cool

tonight, I

PubSubHubbub test