This is Dan Pritchett's Typepad Profile.
Join Typepad and start following Dan Pritchett's activity
Join Now!
Already a member? Sign In
Dan Pritchett
San Jose, CA
Software is what I do and have for most of my life.
Interests: Software architecture, Java, reputation, identity, scalable SOA, semantics, mobile
Recent Activity
Many years ago at a management training class the instructor went over the 4 stages of knowledge progression. I found it a reasonable perspective. They are: We don't know we don't know. We know we don't know. We don't know... Continue reading
During one of my first few weeks at eBay I got involved in a conversation about mark down logic. Now, I had only been at the company for a short while and I was working for an e-commerce company, so... Continue reading
Database storage is expensive. This is especially true if you build a traditional SAN based M+N cluster. The cost of the storage array, fiber channel switches, fiber channel interfaces, drives the cost per terabyte into the thousands quite easily. And... Continue reading
Follow the technology business news and barely a day goes by that somebody doesn't announce a new or refined social commerce or recommendation product. The concept is quite simple. Use my social graph to filter goods, services, and content making... Continue reading
If you ask people for their recommended reading on agile, you'll probably get a list of books on agile process, blogs by agile experts, and social media recommendations to follow that will be full of agile coaches. And, I wouldn't... Continue reading
A regular cadence of posting has never been my forte with this blog. Part of it is because, well, writing isn't my first passion and the rest of it is that I made a commitment to try and maintain quality.... Continue reading
The age of highly personal computing is emerging. Follow any major technology news source and you will hear about semantics, relevance, and intelligent agents almost daily. The emergence of tremendous compute resources is making it possible for us to finally... Continue reading
Nope, but I will add it to my reading list. Thanks!
Yes, let's kill productivity! Okay, not productivity but all attempts to measure the productivity of software engineers. It is a flawed, pointless, futile attempt to put metrics on something that is immeasurable. The energy spent on this could be applied... Continue reading
How many of you have heard of Conway's Law? Melvin Conway postulated in 1968 that: ...organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations. I usually paraphrase it into:... Continue reading
We might consider posting it but one of the challenges is what is outlined in that ticket. How do you rendezvous with the Flume agent. We've picked a very specific policy for how to do it which may not be a policy that every organization is willing to adopt.
I've been pondering the problem of how does an organization achieve innovation and at the same time have some level of consistency in the patterns and components that are in use across teams. Considering the very nature of innovation is... Continue reading
Foursquare had a well discussed outage last week. This wasn't good news for Foursquare but it can be for the rest of us. By looking at what happened, we can all learn steps we can take to avoid a similar... Continue reading
Yes, you could use a decorator to parse your log file and get back to the structure. I could see where that can become more problematic if you are setting properties on the log record in log4j, but of course it could be done that way. The appender does save me from having to manage log files on the application server, which is a primary motivator for using Flume to begin with. And yes, Flume can be used on as few as 1 machines.:)
We wrote an appender to get better structured content into Flume. If we simply tail the log file then we lose content (unless I'm missing something) like level, parsed timestamp, class name, thread name, etc. As written the appender also picks up location information and properties if present and places that into fields. The purpose being to keep as much logger information structured as possible. The HBASE sink sounds great. I'll have a look!
I've been looking at Flume in depth and it is a very powerful and useful platform that solves several operational problems at once. And the best part is that it is surprisingly simple to use and understand. Application logs are... Continue reading
Is it NoSQL or NOSQL (Not Only SQL)? Or is it SQL on MySQL, Postgres, or Oracle? Or maybe, just files? My simple answer is most likely any and all, once you go through the process of defining your requirements... Continue reading
I believe architects need to contribute some code but it will vary to the amount. It helps to keep them informed about the challenges of crafting the system. Deployment is clearly part of the architecture but I don't separate it from the code. Deployability and manageability are part of delivering a complete solution.
This shouldn't start any spirited debate. Attempting to define software architecture. It's only been debated and discussed for as long as the title "Software Architect" has existed, but recently, it's become a curious role. As software processes compress, relying on... Continue reading
Dan Pritchett is now following The Typepad Team
Mar 15, 2010