Hello everyone ! I have a problem in my elk environment. Follows:
- nginx log:
124.134.229.30 - - [18/Aug/2015:03:30:01 +0800] "POST /?nav=24xsc&flaglot=hnquick5&tit=%E6%B2%B3%E5%86%855%E5%88%86%E5%BD%A9 HTTP/1.0"200 23 lotteryid=14&issue=20150818-043&flag=gettime "http://x8shi.com/?nav=24xsc&flaglot=hnquick5&tit=%E6%B2%B3%E5%86%855%E5%88%86%E5%BD%A9""Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.63 Safari/537.36" 124.134.229.30 x8shi.com 0.140 0.027 -
- log format:
log_format access '$remote_addr - $remote_user [$time_local] "$request"' '$status $body_bytes_sent $request_body "$http_referer"' '
"$http_user_agent" $http_x_forwarded_for $host $request_time $upstream_response_time' " $upstream_cache_status";
now , I should how to write this grok pattern for logstash ?
Please help me …
This textbox defaults to using Markdown to format your answer.
You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!
The following grok pattern would match the log message as you resented it:
NGUSERNAME [a-zA-Z\.\@\-\+_%]+
NGUSER %{NGUSERNAME}
%{IPORHOST:clientip} %{NGUSER:ident} %{NGUSER:auth} \[%{HTTPDATE:timestamp}\] \"(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|-)\"%{NUMBER:response} (?:%{NUMBER:bytes}|-) %{NOTSPACE:querystring} (?:"(?:%{URI:referrer}|-)"|%{QS:referrer})%{QS:agent} %{IPORHOST:forwardedfor} %{IPORHOST:host} %{NUMBER:upstreamresponse} (?:-|%{NUMBER:cache})
I was able to put that together using the great Grok Constructor tool. Using its Matcher, you can test and tweak your patterns.
For a deeper dive into writing filters for Logstash, check out:
Become a contributor for community
Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.
DigitalOcean Documentation
Full documentation for every DigitalOcean product.
Resources for startups and SMBs
The Wave has everything you need to know about building a business, from raising funding to marketing your product.
The developer cloud
Scale up as you grow — whether you're running one virtual machine or ten thousand.
Get started for free
Sign up and get $200 in credit for your first 60 days with DigitalOcean.*
*This promotional offer applies to new accounts only.