Jon Gifford on the combination of cloud computing and log management and search applications

The Technology podcast (Podcast) is hosted by Andrew Glover, a passionate and inquisitive man, who will bring you more content and themes that you will relate to and that are also popular in the industry.

The CTO of the loggly service website, Jon Gifford and Andy, explored the concept of log as service (logging as a service) and its help with log management and manipulation. Understanding how to control and manage such a large system in real time is a http://www.aliyun.com/zixun/aggregation/7239.html "> Interesting thing." At the same time, you will learn more about the details of loggly operations, where it differs from other services and products, and its future direction.

Jon Gifford is the CTO and chief architect of Loggly. For a long time, Jon was primarily responsible for writing Lucene-related code for many companies. Jon and his partner are now starting a business in San Francisco.

Here's a look at Andrew Glover interview with Jon Gifford:

GLOVER: This is Andy GLOVER, the Java Technology Series for DeveloperWorks podcasts. The guest of the year is Jon Gifford. He is Loggly's chief technology officer and co-founder. Jon, I think our conversation started when I asked you what a loggly was.

Gifford:loggly is a log-as-service system. That's what we're doing. Everyone has a log. Logs are everywhere. They exist on every machine you own. They contain valuable data and contain some less valuable data. and managing them is a pain.

So, basically, our product is a way to make it easier for you to manage logs, and it provides tools that you can use to accomplish things that you can't do otherwise.

So basically you can think of us as a place where you can place all the logs in all systems and events, and even you can track browser events and things like that directly through us. So this really is ... We really are a big log dump in the sky.

GLOVER: In the sky, yes. Let me ask you one more question. As a result, applications for both log and output logs are already spread around the world, and they are available wherever there is a computer.

GIFFORD: Yes.

GLOVER: But you're relatively new. So, what makes you suddenly decide, well, let's set up a company to solve this problem. Why has the problem not been solved before?

GIFFORD: I think the problem has been solved. There are a lot of companies in the market. In the company that solves this problem for you, Splunk is probably the most famous. What we want to do is try ... Try to build a system that is very easy to use, without the various management and headaches in the use of existing systems. So that's why we're in the clouds. That's why we are a service, not a downloaded software or device ...

GLOVER: Good.

GIFFORD: ... These are also common ways to solve this problem. But for me personally, the reason I'm really motivated to do this is because I've been working on a search for a long time; about 15 years. and handled a variety of large systems, high-end up to about 900 machines, and at the low end, for effective products, in fact, loggly may be the smallest product I have used in the production environment, it applies to 15 to 20 machines. So when you are dealing with so many machines, whether in the cloud or in your own "Colo" [or "Coloc"; colocation center ... Editor's note. , you need to know what's going on. So the thing I've been doing in the end seems to be writing this system for myself to monitor all these boxes and drive the dashboard and all the other things you want to do to make sure that you as a developer can stay sane, or that if you have operators, keep them sane.

So, for me, it's like, I've got this, I've been thinking about it a lot of times, it's just like, you know, if I can do it this time, I don't have to do it anymore.

GLOVER: That's it.

GIFFORD: So, from a certain point of view, I think it's because I'm lazy.

GLOVER: Isn't that the definition of what we're developers? Are we lazy? Laugh]

GIFFORD: Yes. Is。 So the other thing about what we're doing, from a purely technical point of view, is that it's a fairly complex distributed system and I like to work on those systems. We use our own system ... Or, we record our own system and record it ourselves. Laugh]

In fact, to check the status of our production servers, we need to look at a loggly instance where all the logs from all production instances come to this instance. In fact, we run it as a separate instance, which seemed a bit unusual before. But basically, when we first started, the code wasn't as stable as it used to be, and I don't think you know that. Laugh]

If something fails in one of our boxes, we occasionally remove it from production. We will move it to its own instance and everything will be normal. Therefore, building highly scalable systems is a very interesting technical challenge. I think I have to say that I particularly love to search as a way of studying data.

GLOVER: Yes.

GIFFORD: I think many of the systems I've built before have been a bit more focused on metrics. That's great. It tells me the implementation of the various tasks. But when there is a problem, it does not necessarily give me much help, in fact I want to find out what caused the problem.

And I think search is a particularly powerful way to solve this problem. So every time using grep and awk, and all those magical ... Perl and Python, and Ruby and all the magic things that you can use to extract information from text. But I think, in some ways, I think just pure naked search has a lot of benefits that you use all those other things you can't get.

GLOVER: I want to ask a question, in fact, I think of two questions. I hope I will remember them. The first is that, as you mentioned, it is a log as a service in the cloud.

GIFFORD: Yes.

GLOVER: So what does this mean for me as a developer? How do I use loggly? How does it work?

GIFFORD: Good. Basically, we have several different ways to get data into the system. If you are a developer and you want to send us data, you can use Syslog, and for most people, at least most UNIX developers are syslog Appender. It is a very simple protocol, for example, you can execute it on Java, and you can execute it with a Syslog Appender.

In other languages, there are other ways to log directly to Syslog. If you are prepared to do so, you only need to set up your Syslog Appender on the case you want to record and forward the logs to us. This is fairly simple and is a very, very lightweight way to solve the problem.

GLOVER: When you say Syslog appender, I mean, it's like log4j, right? I just need to configure log4j?

Gifford:log4j. Is。

GLOVER: That's right. This is a common log library for 99% of existing Java applications. Laugh]

GIFFORD: That's right.