Any BI or log tools to import dumb file [closed]












0















Currently we do not have any log tools for filtering a huge dumb file. Is there any open source BI or log tool able to import a dumb file and do auto/selectable filtering?



Current dumb file example:



2019-02-11 03:06:30.202  INFO maskParams=false

2019-02-11 03:06:30.205  INFO Start request :: ipAddress=103.22.200.95

2019-02-11 03:06:30.205  INFO  User-Agent = Mozilla/5.0 (Linux; U; Android 7.1.1; zh-cn; MI MAX 2 Build/NMF26F) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/61.0.3163.128 Mobile 

Safari/537.36 XiaoMi/MiuiBrowser/10.5.1

2019-02-11 03:06:30.205  INFO  Request Query String=[customer_name]

2019-02-11 03:06:30.335  INFO  Checking current ip address

2019-02-11 03:06:30.335  ERROR Not a valid ip address 2408:84e4:412:4b02:73cd:9684:1ab0:8437

2019-02-11 03:06:30.336  INFO  Get country from GeoIpRange, ip =2408:84e4:412:4b02:73cd:9684:1ab0:8437

2019-02-11 03:06:30.336  ERROR Not a valid ip address 2408:84e4:412:4b02:73cd:9684:1ab0:8437

2019-02-11 03:06:30.385  INFO  End request (00:00:00.183)


After importing the dumb file, able to provide me the below details:




  • Start Date and time/

  • End Date and time/

  • customer name/

  • no valid ip address details






text-processing logs







share|improve this question















closed as too broad by Jeff Schaller, Christopher, Sparhawk, jimmij, Mr Shunz Feb 15 at 10:32


Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.























    0















    Currently we do not have any log tools for filtering a huge dumb file. Is there any open source BI or log tool able to import a dumb file and do auto/selectable filtering?



    Current dumb file example:



    2019-02-11 03:06:30.202  INFO maskParams=false
    
2019-02-11 03:06:30.205  INFO Start request :: ipAddress=103.22.200.95
    
2019-02-11 03:06:30.205  INFO  User-Agent = Mozilla/5.0 (Linux; U; Android 7.1.1; zh-cn; MI MAX 2 Build/NMF26F) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/61.0.3163.128 Mobile 
    
Safari/537.36 XiaoMi/MiuiBrowser/10.5.1
    
2019-02-11 03:06:30.205  INFO  Request Query String=[customer_name]
    
2019-02-11 03:06:30.335  INFO  Checking current ip address
    
2019-02-11 03:06:30.335  ERROR Not a valid ip address 2408:84e4:412:4b02:73cd:9684:1ab0:8437
    
2019-02-11 03:06:30.336  INFO  Get country from GeoIpRange, ip =2408:84e4:412:4b02:73cd:9684:1ab0:8437
    
2019-02-11 03:06:30.336  ERROR Not a valid ip address 2408:84e4:412:4b02:73cd:9684:1ab0:8437
    
2019-02-11 03:06:30.385  INFO  End request (00:00:00.183)


    After importing the dumb file, able to provide me the below details:




    • Start Date and time/

    • End Date and time/

    • customer name/

    • no valid ip address details






    text-processing logs







    share|improve this question















    closed as too broad by Jeff Schaller, Christopher, Sparhawk, jimmij, Mr Shunz Feb 15 at 10:32


    Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.





















      0












      0








      0








      Currently we do not have any log tools for filtering a huge dumb file. Is there any open source BI or log tool able to import a dumb file and do auto/selectable filtering?



      Current dumb file example:



      2019-02-11 03:06:30.202  INFO maskParams=false
      
2019-02-11 03:06:30.205  INFO Start request :: ipAddress=103.22.200.95
      
2019-02-11 03:06:30.205  INFO  User-Agent = Mozilla/5.0 (Linux; U; Android 7.1.1; zh-cn; MI MAX 2 Build/NMF26F) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/61.0.3163.128 Mobile 
      
Safari/537.36 XiaoMi/MiuiBrowser/10.5.1
      
2019-02-11 03:06:30.205  INFO  Request Query String=[customer_name]
      
2019-02-11 03:06:30.335  INFO  Checking current ip address
      
2019-02-11 03:06:30.335  ERROR Not a valid ip address 2408:84e4:412:4b02:73cd:9684:1ab0:8437
      
2019-02-11 03:06:30.336  INFO  Get country from GeoIpRange, ip =2408:84e4:412:4b02:73cd:9684:1ab0:8437
      
2019-02-11 03:06:30.336  ERROR Not a valid ip address 2408:84e4:412:4b02:73cd:9684:1ab0:8437
      
2019-02-11 03:06:30.385  INFO  End request (00:00:00.183)


      After importing the dumb file, able to provide me the below details:




      • Start Date and time/

      • End Date and time/

      • customer name/

      • no valid ip address details






      text-processing logs







      share|improve this question
















      Currently we do not have any log tools for filtering a huge dumb file. Is there any open source BI or log tool able to import a dumb file and do auto/selectable filtering?



      Current dumb file example:



      2019-02-11 03:06:30.202  INFO maskParams=false
      
2019-02-11 03:06:30.205  INFO Start request :: ipAddress=103.22.200.95
      
2019-02-11 03:06:30.205  INFO  User-Agent = Mozilla/5.0 (Linux; U; Android 7.1.1; zh-cn; MI MAX 2 Build/NMF26F) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/61.0.3163.128 Mobile 
      
Safari/537.36 XiaoMi/MiuiBrowser/10.5.1
      
2019-02-11 03:06:30.205  INFO  Request Query String=[customer_name]
      
2019-02-11 03:06:30.335  INFO  Checking current ip address
      
2019-02-11 03:06:30.335  ERROR Not a valid ip address 2408:84e4:412:4b02:73cd:9684:1ab0:8437
      
2019-02-11 03:06:30.336  INFO  Get country from GeoIpRange, ip =2408:84e4:412:4b02:73cd:9684:1ab0:8437
      
2019-02-11 03:06:30.336  ERROR Not a valid ip address 2408:84e4:412:4b02:73cd:9684:1ab0:8437
      
2019-02-11 03:06:30.385  INFO  End request (00:00:00.183)


      After importing the dumb file, able to provide me the below details:




      • Start Date and time/

      • End Date and time/

      • customer name/

      • no valid ip address details






      text-processing logs




      text-processing logs






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Feb 14 at 13:46









      fra-san

      1,8151518




      1,8151518










      asked Feb 14 at 12:57









      julian leejulian lee

      51




      51




      closed as too broad by Jeff Schaller, Christopher, Sparhawk, jimmij, Mr Shunz Feb 15 at 10:32


      Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.









      closed as too broad by Jeff Schaller, Christopher, Sparhawk, jimmij, Mr Shunz Feb 15 at 10:32


      Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
























          1 Answer
          1






          active

          oldest

          votes


















          0














          There are many open source log collectors and indexers:



          Splunk? Elasticsearch? Logstash? Kibana?



          Are they BI tools? I'm not sure. Only if you plot some business metrics I guess.






          share|improve this answer
























          • i had tested Splunk and it really give me what I wanted. Thanks!

            – julian lee
            Feb 15 at 3:02


















          1 Answer
          1






          active

          oldest

          votes








          1 Answer
          1






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes









          0














          There are many open source log collectors and indexers:



          Splunk? Elasticsearch? Logstash? Kibana?



          Are they BI tools? I'm not sure. Only if you plot some business metrics I guess.






          share|improve this answer
























          • i had tested Splunk and it really give me what I wanted. Thanks!

            – julian lee
            Feb 15 at 3:02
















          0














          There are many open source log collectors and indexers:



          Splunk? Elasticsearch? Logstash? Kibana?



          Are they BI tools? I'm not sure. Only if you plot some business metrics I guess.






          share|improve this answer
























          • i had tested Splunk and it really give me what I wanted. Thanks!

            – julian lee
            Feb 15 at 3:02














          0












          0








          0







          There are many open source log collectors and indexers:



          Splunk? Elasticsearch? Logstash? Kibana?



          Are they BI tools? I'm not sure. Only if you plot some business metrics I guess.






          share|improve this answer













          There are many open source log collectors and indexers:



          Splunk? Elasticsearch? Logstash? Kibana?



          Are they BI tools? I'm not sure. Only if you plot some business metrics I guess.







          share|improve this answer












          share|improve this answer



          share|improve this answer










          answered Feb 14 at 15:35









          MarcelMarcel

          1865




          1865













          • i had tested Splunk and it really give me what I wanted. Thanks!

            – julian lee
            Feb 15 at 3:02



















          • i had tested Splunk and it really give me what I wanted. Thanks!

            – julian lee
            Feb 15 at 3:02

















          i had tested Splunk and it really give me what I wanted. Thanks!

          – julian lee
          Feb 15 at 3:02





          i had tested Splunk and it really give me what I wanted. Thanks!

          – julian lee
          Feb 15 at 3:02



          -

          Popular posts from this blog

          How to make a Squid Proxy server?

          Image
          1 I want to set up squid proxy server for monitoring user activity in a network and Internet Web URL filtering for user. I also want to configure the user login, user log, user access group, and block unwanted web sites in my network. Please help me and guide me the full process to do this. server proxy squid share | improve this question edited Jul 31 '12 at 14:45 Jorge Castro 37.1k 107 422 617 asked Jul 31 '12 at 5:16 Goivind Tiwari Goivind Tiwari
          Read more

          Is this a new Fibonacci Identity?

          Image
          6 1 $begingroup$ I have found the following Fibonacci Identity (and proved it). If $F_n$ denotes the nth Fibonacci Number, we have the following identity begin{equation} F_{n-r+h}F_{n+k+g+1} - F_{n-r+g}F_{n+k+h+1} = (-1)^{n+r+h+1} F_{g-h}F_{k+r+1} end{equation} where $F_1 = F_2 = 1$ , $r leq n$ , $h leq g$ , and $n, g, k in mathbb{N}$ . It is not too hard to show that this identity subsumes Cassini's Identity, Catalan's Identity, Vajda's Idenity, and d'Ocagne's identity to name a few. I have done a pretty thorough literature review, and I have not found anything like this, but I am still wondering if anyone has seen this identity before? I found this by accident after noticing some patterns in some analysis work I was doing, so if this is already known I would be curious to see w
          Read more

          Touch on Surface Book

          Image
          0 How can I make touch work on my Surface Book? About my Surface Book: Model: Intel Core i7-6600 U @ 2.60 GHz with 8 GB Ram and Nvidia GPU OS: Ubuntu 17.04 Latest, kernel 4.10.0-13-generic System outputs on Ubuntu: ~$ xinput list ⎡ Virtual core pointer id=2 [master pointer (3)] ⎜ ↳ Virtual core XTEST pointer id=4 [slave pointer (2)] ⎜ ↳ USB OPTICAL MOUSE id=8 [slave pointer (2)] ⎜ ↳ Microsoft Surface Keyboard Consumer Control id=10 [slave pointer (2)] ⎜ ↳ Microsoft Surface Keyboard Touchpad id=11 [slave pointer (2)] ⎣ Virtual core keyboard id=3 [master keyboard (2)] ↳ Virtual core XTEST keyboard id=5 [slave keyboard (3)]
          Read more