github.com/matomo-org/matomo-log-analytics commits

Another short at fixing the test

4e77f98b8fb4f3b78dc3c65711ff73179a99d1e5 authored almost 11 years ago by mattab <[email protected]>

refs #4564 we have to get the token from the piwik_tests database

cf80b12ac9e349ddcb939b98baa99cbc67de8eb6 authored almost 11 years ago by Thomas Steur <[email protected]>

refs #4564 fix import logs and archive.sh did no longer work because there is no longer a superuser in the config. Read directly the tokenauth of any superuser from a generated file instead. The updatetoken.php will create a file containing the needed token in tmp/cache which will not be served by default (on apache). Also the script contains directly an exit to avoid execution or anything from the browser or cli

50483ff72ce59ad0d97162205d17c78157ebca9a authored almost 11 years ago by Thomas Steur <[email protected]>

Exclude Baidu

Chinese market leading search engine.

24331cb59662fb987600c7ae2504e26d8f5f7bb2 authored almost 11 years ago by Daniel Aleksandersen <[email protected]>

import_logs: added include_path options.

fb418f817515bf95a6996669eb7052ec965ed70d authored almost 11 years ago by cbay <[email protected]>

Refs #4352 increase to 100,000, it takes ~ 6 seconds on my box on a file with 100k lines, so still fast enough to fail. If a file has more than 100k wrong lines there's defnitely something wrong with it.

4fd1a0193921e2293fb90204e49d5ea9aec31387 authored about 11 years ago by mattab <[email protected]>

Fixes #4353, quick fix for parsing invalid log lines at the top of the file in the log import script: try first 1000 lines before giving up entirely.

a6442de93a523c0e324f30c5803c0e0b2542c72b authored about 11 years ago by diosmosis <[email protected]>

Minor messages

756942ffb584c79a1faac18b4b48cad4a3bc980c authored about 11 years ago by mattab <[email protected]>

Adding tests with invalid request (see prev commit)

618616f72d876b6aad3dacce135af2ecf85a025c authored about 11 years ago by mattab <[email protected]>

Fixes ValueError: invalid literal for int() with base 10: '-' http://forum.piwik.org/read.php?2,106854

Reverting part of https://github.com/piwik/piwik/pull/120/files

29d9b821402867e4dd830701ba73a1ab292f3e7f authored about 11 years ago by mattab <[email protected]>

Fix tests

1b85ef13ff008000dda2158d900cda694cabd7f3 authored about 11 years ago by mattab <[email protected]>

unit tests for nginx_json format

013183220b3f0e76fc8cb4576c4206fab248fc9c authored about 11 years ago by Gregory Duchatelet <[email protected]>

[Fix] as asked, moved this README update at the end

2148007a213809613ed81bb71a54ca48691645f6 authored about 11 years ago by Gregory Duchatelet <[email protected]>

vim config for tabs with 4 spaces

c6a9d6180464788e1ecf95bc335dbfcc0935c509 authored about 11 years ago by Gregory Duchatelet <[email protected]>

[Fix] parent initialization

2f691bc0f5ea62473d7bf2a164b3db7e57d04aa0 authored about 11 years ago by Gregory Duchatelet <[email protected]>

Add JsonFormat to handle json logs without regex parsing.

Used to live import nginx logs to piwik.

7767488eb135a4bd84f882ab94175bd771b204c1 authored about 11 years ago by Gregory Duchatelet <[email protected]>

Adding .xsd extension to file list, suggested in forum http://forum.piwik.org/read.php?2,106633

1935979455fa31e734d378643bfb5762d5f5aead authored about 11 years ago by mattab <[email protected]>

Try fixing travis build (by making Db connection lazily created singleton).

4925babe6584471369811c63ea40b1efd9b36a1f authored about 11 years ago by Benaka Moorthi <[email protected]>

debugging ui tests failure on travis

1790719ea5198620ab58f8dcb9694bf5f35b53fd authored about 11 years ago by Benaka Moorthi <[email protected]>

Fixes #4045, fail log importer if we cannot match a format to the first line of the logfile.

fc5bb32f6e2d0cdc4c1d5ddfab4b8e7a2ddf140b authored about 11 years ago by Benaka Moorthi <[email protected]>

Revert debugging change.

0294614b837935165f330a38d4d8234887dcfc93 authored over 11 years ago by Benaka Moorthi <[email protected]>

Fixing build.

657a96e7b04cf59cc3570c3adfac140cc54f3844 authored over 11 years ago by Benaka Moorthi <[email protected]>

Minor code changes + style following review

194f33c17d1cf252774d0da741d57fdd9b638d59 authored over 11 years ago by mattab <[email protected]>

Fix stalling tests in ArchiveCronTest

refs #4141

7b3d86df689ac0e1e775346526a846eee0627219 authored over 11 years ago by Fabian Becker <[email protected]>

Revert this change

8cb52a6ed5e2133d24dec800f6e586cb0beb3abd authored over 11 years ago by mattab <[email protected]>

Fix Tracker variable was not set properly

3a8d533cf3469f9fd7f6e44cb4bb94592d313201 authored over 11 years ago by mattab <[email protected]>

Fixes #4072 thanks for the patch

7049a2228d3c16868577b725262b806082b7ea23 authored over 11 years ago by mattab <[email protected]>

Checking if format is not nil before logging

17a86b658c8edd70c7490408d8fe6e7a73beda95 authored over 11 years ago by Andreas Thielen <[email protected]>

Refs #3741 PHP Namespace conversion continued: Tracker classes,

renaming Unzip to Uncompress,
Renaming Tracker/Referer to Tracker/Referrer,
and few more

16af9ea2ee296a80d692811652a172365806657e authored over 11 years ago by mattab <[email protected]>

Merge branch 'master' of github.com:piwik/piwik into pleineLune__layout-cleanup

8fa6a87a62a274daf6ae5a5949c7b8c093778e37 authored over 11 years ago by Thomas ZILLIOX <[email protected]>

Merge with master

8ae27ebf01ba921ca6f5b0057d55d5e2ed93ef92 authored over 11 years ago by Thomas ZILLIOX <[email protected]>

Handle log files in log importer where site URL's protocol + host will be prepended to the path even if the path already contains a protocol & host.

80abd51c1c09424b44cdff27ed1751654c1519e4 authored over 11 years ago by Benaka Moorthi <[email protected]>

Refs #1700 important: renaming generation_time_ms to gt_ms for keeping parameters name short.

Updated doc at: http://piwik.org/docs/tracking-api/reference/
and http://piwik.org/docs/page-speed/

7cedb8ef86bc12aeb9634ee8287a08b793edd715 authored over 11 years ago by mattab <[email protected]>

HTTP status codes should be a page scope custom variable.

eae99b2ac3ac6e7255aeefaf2a00543235e20575 authored over 11 years ago by Benjamin Doherty <[email protected]>

Update import_logs.py

Added support to Icecast2 HTTP logs format. NSCA_EXTENDED + SessionTime

d3e223ecd32752e86d43f3db488827230ea67260 authored over 11 years ago by AlejandroF <[email protected]>

Fixes #3832, add --force-lowercase-path option to log importer script that makes all paths lowercase before importing hits.

2dd3468d8988a8f39c2e413feb9c61212e61e39b authored over 11 years ago by diosmosis <[email protected]>

Beautify the time in Metadata reports + tests

Small update to log analytics readme
Fixes #1700

Awesome work Timo!

af39a8057f8a254070dd0eccab10922eeee458f8 authored over 11 years ago by mattab <[email protected]>

refs #1700 performance analytics

* adding avg_time_generation to Actions.get + integration tests
* adding sparkline for average...

a4e63bd172418fe9c548240a4484504e1cfc847a authored over 11 years ago by Timo Besenreuther <[email protected]>

refs #1700 basic performance analytics: handle server generation time for each page and page title

CORE

* formatting sub-second times
* getColumn() method on data table array (in order to beh...

5f4f6289c654968a083fb6be18e834b05de761dd authored over 11 years ago by Timo Besenreuther <[email protected]>

Fixes #3805, reverted change in log importer that looked for end-of-line after format regex match and modified format autodetection logic to pick the format based on whether the format matches and the number of groups returned in the match.

Notes:
* Added several more tests to log importer tests.py. Added tests for checking format of...

943369894895a210a29004eab80894667a11a39b authored almost 12 years ago by diosmosis <[email protected]>

adding some excluded user agents to log import

21edfe12de20df659c48c5825414ea77970562de authored almost 12 years ago by Timo Besenreuther <[email protected]>

Fixed if condition if optoions replay_tracking enabled.

367a944fe490fd593699fe0aa145d3a047c97006 authored almost 12 years ago by Tomek Wójcik <[email protected]>

Fixed 'rec' param value when replay-tracking enabled

a5df1c907cf1d6e6d28d73afe35b50e356f6daaa authored almost 12 years ago by Tomek Wójcik <[email protected]>

1.11-b2

8ccc9b279eeb1f0c3c22cd6b08c4adb470e2e399 authored almost 12 years ago by mattab <[email protected]>

Fixes #3632, refactor/tweak pull request #34 merge.

1134d8dcde1cca9d09bfcacf2fddbcd0f6bdd743 authored almost 12 years ago by diosmosis <[email protected]>

Refs #3632, Remove stray debugging code.

e2fa33446faaaeb1ead4f631b535ac89abd60740 authored almost 12 years ago by diosmosis <[email protected]>

Refs #3632, add integration test for import script replay tracking.

2dcd741eead159cf3f8e5039e8c5b24ed6b352c1 authored almost 12 years ago by diosmosis <[email protected]>

Fixing build (make sure regex format checks for end of line in log import script).

1170e172e30aef3e35b95c9e840e5169a059cba7 authored almost 12 years ago by diosmosis <[email protected]>

Merge pull request #34 from clearcode/master

Modify replay tracking to use site ID in query string when using replay-tracking option.

af2aafbff6160ec740261ed1a73c80ecaf5dbbc6 authored almost 12 years ago by diosmosis <[email protected]>

Create sample log file using Amazon S3 log format

3ef9e82e3e36a6a9bb6a0d0c575072ee059df5c2 authored almost 12 years ago by lesensei <[email protected]>

Add Amazon S3 log format to import_logs.py

8621fad2163e7381bdce84532f77d84ff3a99dd2 authored almost 12 years ago by lesensei <[email protected]>

Fixed if condition.

c86032883dc40a71a9fab9ebe31a225fc1650927 authored almost 12 years ago by Tomek Wójcik <[email protected]>

Fixed replay-tracking option to parse different sites ID.

c92440a58ecbad80cf6f224181c3a367569d5fd7 authored almost 12 years ago by Tomek Wójcik <[email protected]>

Merge pull request #28 from clearcode/master

Adds replay-tracking option to import_logs script.

Thanks @clearcode!

e666e52176210d368a6e6e26675947c28c261a82 authored almost 12 years ago by Fabian Becker <[email protected]>

Reverted static methods to instance methods.

2a5aeb0a98cecc60af881916d2939114bccc4d46 authored almost 12 years ago by Tomek Wójcik <[email protected]>

Rename README to README.md and added markdown markup to make it look better on Github

[ci skip]

2275e560e3a1e5b5454e68ce7bdf19d206d325e2 authored almost 12 years ago by Fabian Becker <[email protected]>

Handling UnicodeDecodeErrors that may be raised while processing logs.

f27df8257e4b405b3bbaba01c3c0b299c9c53cde authored almost 12 years ago by Tomek Wójcik <[email protected]>

Adds replay-tracking option to import_logs script.

663ed22ff8d5e9c6b323c5a9a631dcdbbc65da80 authored almost 12 years ago by Tomek Wójcik <[email protected]>

Fixes #3655 thx for the report!

git-svn-id: http://dev.piwik.org/svn/trunk@7778 59fd770c-687e-43c8-a1e3-f5a4ff64c105

f1990a10f190a92cda3b0c09eecd53b8c0605754 authored almost 12 years ago by mattpiwik <[email protected]>

Fixing build, intermediate commit, add one more debugging call for failure to get site URL.

git-svn-id: http://dev.piwik.org/svn/trunk@7731 59fd770c-687e-43c8-a1e3-f5a4ff64c105

502529496e9e3cd5eafb4d84e75613427d74af32 authored almost 12 years ago by benakamoorthi <[email protected]>

Ref #3632: log-analytics: add Hit.args to let the parser override specific args.

git-svn-id: http://dev.piwik.org/svn/trunk@7693 59fd770c-687e-43c8-a1e3-f5a4ff64c105

b4455bfc55a827135c80a0cd681a4acb7f5ae460 authored almost 12 years ago by cbay <[email protected]>

Fixes #3548 Refs #3163

Any visitor with a user agent containing "spider" will be classified a bot

git-svn-id: http://de...

0c6b1b8a8aad1e67d23cc96dcb842e42778b80e6 authored about 12 years ago by mattpiwik <[email protected]>

import_logs: minor refactoring.

git-svn-id: http://dev.piwik.org/svn/trunk@7404 59fd770c-687e-43c8-a1e3-f5a4ff64c105

5fec76f47d91307963b67179212d152dadc246ff authored about 12 years ago by cbay <[email protected]>

Refs #3163: don't fail to autodetect the format for empty files.

git-svn-id: http://dev.piwik.org/svn/trunk@7383 59fd770c-687e-43c8-a1e3-f5a4ff64c105

67b76d27c04ffa6e31b9fb7f8b934f9e1c9d1755 authored about 12 years ago by cbay <[email protected]>

Refs #3163: Log Parser README Update with Nginx Log Format for Common Complete, thanks to phikai.

git-svn-id: http://dev.piwik.org/svn/trunk@7382 59fd770c-687e-43c8-a1e3-f5a4ff64c105

06f404cdd81465685766c2dac1daaebfb86f7dc5 authored about 12 years ago by cbay <[email protected]>

import_logs.py: added option --show-progress-delay.

git-svn-id: http://dev.piwik.org/svn/trunk@7381 59fd770c-687e-43c8-a1e3-f5a4ff64c105

27eeca75a9b136c0e41a3911a59aaac56439366b authored about 12 years ago by cbay <[email protected]>

import_logs.py: lowercase the host and strip dots at both ends.

git-svn-id: http://dev.piwik.org/svn/trunk@7380 59fd770c-687e-43c8-a1e3-f5a4ff64c105

3b88509817ea8ebbacf5167a5b373440a5d5937e authored about 12 years ago by cbay <[email protected]>

import_logs.py: set the Queue size to 2 rather than 10000 since we don't store hits anymore, but a bunch of hits.

git-svn-id: http://dev.piwik.org/svn/trunk@7379 59fd770c-687e-43c8-a1e3-f5a4ff64c105

4171ebbb3f443525da722d44e66b81e296ce9f5e authored about 12 years ago by cbay <[email protected]>

import_logs.py: optimization: only parse the date if the hit is not excluded.

git-svn-id: http://dev.piwik.org/svn/trunk@7378 59fd770c-687e-43c8-a1e3-f5a4ff64c105

ecce6a1850be062ab1987ba93854f7202fc8b67f authored about 12 years ago by cbay <[email protected]>

import_logs.py: optimizations.

git-svn-id: http://dev.piwik.org/svn/trunk@7377 59fd770c-687e-43c8-a1e3-f5a4ff64c105

19a8fd6ba48084ee7412fd8c7cb4778d90519736 authored about 12 years ago by cbay <[email protected]>

import_logs.py: added the --invalidate-dates option.

git-svn-id: http://dev.piwik.org/svn/trunk@7376 59fd770c-687e-43c8-a1e3-f5a4ff64c105

43c7e02637e4bc284dc5c2ff0ff874bfd53a69cd authored about 12 years ago by cbay <[email protected]>

import_logs.py: some refactoring to make it easier to reuse the script.

git-svn-id: http://dev.piwik.org/svn/trunk@7375 59fd770c-687e-43c8-a1e3-f5a4ff64c105

5b5098c9b2b58514f1800a16e12572c7426bac3e authored about 12 years ago by cbay <[email protected]>

Refs #3163 Adding libwww in excluded user agents, since libwww-perl is a common bot

As reported in: http://forum.piwik.org/read.php?3,95844

git-svn-id: http://dev.piwik.org/svn/tru...

c032cf0c7ca4def625625967535b6b71325a873b authored about 12 years ago by mattpiwik <[email protected]>

Added --disable-bulk-tracking and --debug-force-one-hit-every-Ns=N options to log importer.

git-svn-id: http://dev.piwik.org/svn/trunk@7030 59fd770c-687e-43c8-a1e3-f5a4ff64c105

8fedf6bcb189ee0293e052f08912578a79bacd70 authored over 12 years ago by benakamoorthi <[email protected]>

Fixes #3346, check that elapsed time is non-zero before calculating average import speed.

git-svn-id: http://dev.piwik.org/svn/trunk@6876 59fd770c-687e-43c8-a1e3-f5a4ff64c105

bbf8023721f8131d7b9130ec90914b174aea2407 authored over 12 years ago by benakamoorthi <[email protected]>

Fix tab in log importer script.

git-svn-id: http://dev.piwik.org/svn/trunk@6827 59fd770c-687e-43c8-a1e3-f5a4ff64c105

affbc935544ebb56f764330f8d34f9cd6430206b authored over 12 years ago by benakamoorthi <[email protected]>

Refs #3163, fix concurrency bug in import script where sites get created more than once when --add-sites-new-hosts is used.

git-svn-id: http://dev.piwik.org/svn/trunk@6824 59fd770c-687e-43c8-a1e3-f5a4ff64c105

d765a409ca655ebfd48fa41fdfb3925b155becd9 authored over 12 years ago by benakamoorthi <[email protected]>

Refs #3163, show average records/s along w/ current records/s in log importer.

git-svn-id: http://dev.piwik.org/svn/trunk@6756 59fd770c-687e-43c8-a1e3-f5a4ff64c105

38ef488802ed9c630b4938b320bb3e368f2cca0d authored over 12 years ago by benakamoorthi <[email protected]>

Misc minor

git-svn-id: http://dev.piwik.org/svn/trunk@6755 59fd770c-687e-43c8-a1e3-f5a4ff64c105

f40e9124d899e82eef4ac9b42ebc5a33ce7d0788 authored over 12 years ago by mattpiwik <[email protected]>

more debug

git-svn-id: http://dev.piwik.org/svn/trunk@6753 59fd770c-687e-43c8-a1e3-f5a4ff64c105

a66ecc09c00fd070b7818892e398b52e565c7f0b authored over 12 years ago by mattpiwik <[email protected]>

Tell me more of the error

git-svn-id: http://dev.piwik.org/svn/trunk@6751 59fd770c-687e-43c8-a1e3-f5a4ff64c105

9e3ab805db69f1c22f3cb3034a8bc6d24085f22f authored over 12 years ago by mattpiwik <[email protected]>

Refs #3163

* removing tracker-url param
* fixing build?

git-svn-id: http://dev.piwik.org/svn/trunk@6749 5...

cecd3889be7f43fd9ce42fe1c6014d7f8a761043 authored over 12 years ago by mattpiwik <[email protected]>

Improving error msg as per http://forum.piwik.org/read.php?2,92105

git-svn-id: http://dev.piwik.org/svn/trunk@6746 59fd770c-687e-43c8-a1e3-f5a4ff64c105

0c7fc0c149e0d89c4948bbc734349c7da0e9e46e authored over 12 years ago by mattpiwik <[email protected]>

Refs #3163, made size of parsing chunk == to max payload size * recorder count.

git-svn-id: http://dev.piwik.org/svn/trunk@6740 59fd770c-687e-43c8-a1e3-f5a4ff64c105

f075a2c6fb2902516b6d10415497e15b1f12f0e5 authored over 12 years ago by benakamoorthi <[email protected]>

Refs #3163 - clarifying this option shouldn't be used by default

git-svn-id: http://dev.piwik.org/svn/trunk@6739 59fd770c-687e-43c8-a1e3-f5a4ff64c105

9d3499634f05ea641ddf4184310b299bce6a18ed authored over 12 years ago by mattpiwik <[email protected]>

Refs #3163, modified log importer to use bulk tracking capability.

Notes:
* Added 'ua' & 'lang' tracker parameters to override user agent & language present in H...

63382283fa6f9537662b077c68bdc4e7300533c4 authored over 12 years ago by benakamoorthi <[email protected]>

Refs #3163, add integration tests (in PHP) for log importer.

git-svn-id: http://dev.piwik.org/svn/trunk@6734 59fd770c-687e-43c8-a1e3-f5a4ff64c105

0651a2124dc95547df61ef62f7188965af85d18b authored over 12 years ago by benakamoorthi <[email protected]>

Refs #3163 updated README to suggest increasing the --recorders value.

git-svn-id: http://dev.piwik.org/svn/trunk@6509 59fd770c-687e-43c8-a1e3-f5a4ff64c105

c54148d4c0a1113550819a3e9531c7eb4f8b372c authored over 12 years ago by cbay <[email protected]>

Refs #3163 Added a reference in README to the Debian/Ubuntu default vhost_combined, thanks aseques.

git-svn-id: http://dev.piwik.org/svn/trunk@6475 59fd770c-687e-43c8-a1e3-f5a4ff64c105

c7fcc15a1ffd2722e5d9756a82ef8bdc9fa20c2f authored over 12 years ago by Cyril Bay <[email protected]>

Refs #3163 Added the --log-hostname option, thanks to aseques.

git-svn-id: http://dev.piwik.org/svn/trunk@6474 59fd770c-687e-43c8-a1e3-f5a4ff64c105

b43219f03fc96d976d4320ede8293108e2a87280 authored over 12 years ago by Cyril Bay <[email protected]>

Refs #3163 Fixed regexp in README.

git-svn-id: http://dev.piwik.org/svn/trunk@6471 59fd770c-687e-43c8-a1e3-f5a4ff64c105

8493c471e392d4fab562549840a43df1f0b8aeaf authored over 12 years ago by Cyril Bay <[email protected]>

Refs #703 Set bots=1 accordingly.

git-svn-id: http://dev.piwik.org/svn/trunk@6433 59fd770c-687e-43c8-a1e3-f5a4ff64c105

0c80ffc7f50d5adab61b6451460e6a94693c348e authored over 12 years ago by Cyril Bay <[email protected]>

Refs #703 Fixed encoding bug with a non-ASCII user-agent.

git-svn-id: http://dev.piwik.org/svn/trunk@6295 59fd770c-687e-43c8-a1e3-f5a4ff64c105

375b77a3e9fb17fe5cca60a22ea13c05a6f2cc75 authored over 12 years ago by Cyril Bay <[email protected]>

Refs #703 Do not append the query string delimiter if there's no query string, thanks reetz.

git-svn-id: http://dev.piwik.org/svn/trunk@6283 59fd770c-687e-43c8-a1e3-f5a4ff64c105

866a79a0e56b36b6428dda46fe6e1337cfdbc53c authored over 12 years ago by Cyril Bay <[email protected]>

Refs #703 Fixed a bug introduced in the IIS parsing refactoring, thanks reetz.

git-svn-id: http://dev.piwik.org/svn/trunk@6282 59fd770c-687e-43c8-a1e3-f5a4ff64c105

5e206392190c21d5e3efc7e958ce084d0bed9b1b authored over 12 years ago by Cyril Bay <[email protected]>

Refs #703 Better handling of query strings.

git-svn-id: http://dev.piwik.org/svn/trunk@6271 59fd770c-687e-43c8-a1e3-f5a4ff64c105

b8b44f596d6c37715a0e16e4a8bf1efef9cbcbea authored over 12 years ago by Cyril Bay <[email protected]>

Refs #703 Fixed an encoding issue with non-ascii paths or referrers.

git-svn-id: http://dev.piwik.org/svn/trunk@6270 59fd770c-687e-43c8-a1e3-f5a4ff64c105

91d80b50404c7fdb9c788d576e3f2da9c7f4099a authored over 12 years ago by Cyril Bay <[email protected]>

Refs #703 Added support for IIS logs, thanks to tiouk.

git-svn-id: http://dev.piwik.org/svn/trunk@6269 59fd770c-687e-43c8-a1e3-f5a4ff64c105

01be7b5cea125bc20059017fba96d9a98afdd641 authored over 12 years ago by Cyril Bay <[email protected]>

Refs #703 Do not crash on encoding errors (thanks invalid_line(line)

git-svn-id: http://dev.piwik.org/svn/trunk@6268 59fd770c-687e-43c8-a1e3-f5a4ff64c105

11e35ed9119293da72cccca6abdf86f9e11bca9b authored over 12 years ago by Cyril Bay <[email protected]>

Refs #703 Typo (renamed file logs/ncsa_extended.log)

git-svn-id: http://dev.piwik.org/svn/trunk@6261 59fd770c-687e-43c8-a1e3-f5a4ff64c105

a31b349ff2050d2c926730b497a2e2a1c544162c authored over 12 years ago by Cyril Bay <[email protected]>

Ecosyste.ms: OpenCollective

github.com/matomo-org/matomo-log-analytics