Ecosyste.ms: OpenCollective

An open API service for software projects hosted on Open Collective.

github.com/HTTPArchive/legacy.httparchive.org

<<THIS REPOSITORY IS DEPRECATED>> The HTTP Archive provides information about website performance such as # of HTTP requests, use of gzip, and amount of JavaScript. This information is recorded over time revealing trends in how the Internet is performing. Built using Open Source software, the code and data are available to everyone allowing researchers large and small to work from a common base.
https://github.com/HTTPArchive/legacy.httparchive.org

standardize on "iphone4" (instead of "inphone"). use curDevice() and allDevices().

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1101 fc7d47d3-c008-acd5-f51f-d19787b8a02f

d573ef9fd84fe24eea8cd101963394b87821223b authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
standardize on "iphone4" (instead of "inphone"). use curDevice() and allDevices().

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1100 fc7d47d3-c008-acd5-f51f-d19787b8a02f

693bb886caebee4c106838db574aa39b46529085 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
standardize on "iphone4" (instead of "inphone"). use curDevice() and allDevices().

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1099 fc7d47d3-c008-acd5-f51f-d19787b8a02f

ba2e5e133b5361c0c807acfaa91016d5e25d18cb authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
Add optional $label param to computeMissingStats to allow updating just ONE crawl. Avoid divide by zero error in correlations. Remove unused copyStatsToTable function.

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1098 fc7d47d3-c008-acd5-f51f-d19787b8a02f

fead380a8255c30018bee1039ca64eb20c30d2d7 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
fix bug: use $gLabel instead of $label. Get minid & maxid from crawl. Only compute stats for THIS crawl.

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1097 fc7d47d3-c008-acd5-f51f-d19787b8a02f

cbf02fe24e39e22056494ed26ab436d6f45a9df6 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
Update the crawls table stats BEFORE computing stats.

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1096 fc7d47d3-c008-acd5-f51f-d19787b8a02f

0c22aa709cd0e45b0f9472127cce7abb2596764f authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
require status.inc

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1095 fc7d47d3-c008-acd5-f51f-d19787b8a02f

aa8141bbd7601c414338d3ae128420b9ef97dc43 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
add status.inc

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1094 fc7d47d3-c008-acd5-f51f-d19787b8a02f

8149f84b71321cda97f62262194ab3ca52f427f8 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
add status.inc. better error checking for available JSON fields.

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1093 fc7d47d3-c008-acd5-f51f-d19787b8a02f

15296c528539d10a229778279fb61d081f9228bf authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
moved code from dbapi.inc here

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1092 fc7d47d3-c008-acd5-f51f-d19787b8a02f

9e6b58046f89365f27a2dcc70fe3d781bbb059e3 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
add support to go to a random website - good for testing when slow queries are cached

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1091 fc7d47d3-c008-acd5-f51f-d19787b8a02f

43104d124a0e9ecb02f243e808bc0c1bedc32d3c authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
take a bunch of code from dbapi.inc

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1090 fc7d47d3-c008-acd5-f51f-d19787b8a02f

f6d0771c1cc20490e07b8d4c570f6734681f75f3 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
moved a bunch of code out into stats.inc, status.inc, and pages.inc

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1089 fc7d47d3-c008-acd5-f51f-d19787b8a02f

23c059f11dd9b7e86dd69aa8a33599db05ee3430 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
code from dbapi.inc

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1088 fc7d47d3-c008-acd5-f51f-d19787b8a02f

554467264bc31ff7568f077c2ee43c1505dad52d authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
move code from dbapi.inc

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1087 fc7d47d3-c008-acd5-f51f-d19787b8a02f

58f12d9286faac12f09e06ae05a44b25fc43c511 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
add crawls.inc

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1086 fc7d47d3-c008-acd5-f51f-d19787b8a02f

d2e5af1a6cfaf77258f14c1d6bc1731cdbb58efc authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
add crawls.inc

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1085 fc7d47d3-c008-acd5-f51f-d19787b8a02f

0c378e45ab297ae87c779b01b92eaec21a34a196 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
change resources.inc to requests.inc

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1084 fc7d47d3-c008-acd5-f51f-d19787b8a02f

cd867f4f5e41b8d855ba63ea0feec19e4d5e4e88 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
add crawls.inc. move a bunch of code to crawls.inc.

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1083 fc7d47d3-c008-acd5-f51f-d19787b8a02f

3706ba6a0065f0224e5b8feb1989dc9f61f850c2 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
add crawls.inc

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1082 fc7d47d3-c008-acd5-f51f-d19787b8a02f

d1f1dd17d3720dc60759144a611b4f4bf4b0181a authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
change resources.inc to requests.inc

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1081 fc7d47d3-c008-acd5-f51f-d19787b8a02f

4ddada32c17773e95c0ee7df6403dfbc5056e2ed authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
add utils.inc

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1080 fc7d47d3-c008-acd5-f51f-d19787b8a02f

1dd59d38c857f4c81f410b636f6f8c50dfe4a388 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
Move a bunch of code to crawls.inc. Add dateRangeCrawls().

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1079 fc7d47d3-c008-acd5-f51f-d19787b8a02f

8359c335a33c4a12b3e8556a32ebdbd29fd4be91 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
add URLs link

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1078 fc7d47d3-c008-acd5-f51f-d19787b8a02f

e6f11da9fbf7eddfd6ca39599cdf92c90dbe02db authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
add URLs link

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1077 fc7d47d3-c008-acd5-f51f-d19787b8a02f

5514f52404316c1bfa500cec2314ba773e54f938 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
moved from resources.inc

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1076 fc7d47d3-c008-acd5-f51f-d19787b8a02f

7d20fcdae256421dfc85e99ae3adcb6bc4c138d0 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
moved to "requests.inc". "resources" in a more appropriate name, but "requests" is used to much in the code it is more intuitive to call this file "requests.inc".

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1075 fc7d47d3-c008-acd5-f51f-d19787b8a02f

b26bccf0a6e93105fa8f2ffd21fc84e94e455adb authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
Improve performance of queries in diffRuns by replace label with pageid range.

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1074 fc7d47d3-c008-acd5-f51f-d19787b8a02f

a4594195ddf8710a21f588ccaf05e3fadcd4ddc7 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
Wrap DEFLATE directive by IfModule to avoid log errors when it gets turned off.

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1073 fc7d47d3-c008-acd5-f51f-d19787b8a02f

eb617abf2261901a5baf9b7adfae79b63904d30e authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
The default mysql storage engine switched from MyISAM to InnoDB in version 5.5.5. We got bit by this. Explicitly set the enging to MyISAM since the InnoDB version of the "pages" table was 4x slower to query.

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1072 fc7d47d3-c008-acd5-f51f-d19787b8a02f

60687de181e562872a653925282792b22c992489 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
Add flushing. If there is an error finding a pageid, redirect.

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1071 fc7d47d3-c008-acd5-f51f-d19787b8a02f

a44eeaf893cc9c665b0e070c56631474b1e4ea14 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
add a comment

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1070 fc7d47d3-c008-acd5-f51f-d19787b8a02f

091be7e9c77426a05aa5cfbd40d50789d113871e authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
add a faster version of archiveLabelsForUrl to make viewsite.php faster. Unfortunately, this returns ALL labels - not just the labels that include the desired URL. So opened a bug (#359) to fix this later.

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1069 fc7d47d3-c008-acd5-f51f-d19787b8a02f

0e7200c770f880d17576bbbd0c5c40322fa80027 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
add a space character - doh!

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1068 fc7d47d3-c008-acd5-f51f-d19787b8a02f

28359a24dd46cf1773a4fb5496a4173dd1fa8cb3 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
add getPrevLabel() to make viewsite.php faster (when calling diffRuns())

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1067 fc7d47d3-c008-acd5-f51f-d19787b8a02f

c43bf5902878f5c9fa0ba92007242bbc3b9e3369 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
tweak function dumpfileName() to include optional table and format

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1066 fc7d47d3-c008-acd5-f51f-d19787b8a02f

e58bbec44fa8dbf539561235fcd8eae49275fc66 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
Use $pageidCond variable. Add check for orphan records. Add new dumpfile code to separate pages from requests and add csv format.

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1065 fc7d47d3-c008-acd5-f51f-d19787b8a02f

e3e92d33ec890f4d45c09f107ef56754427c1648 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
Avoid overwriting NULL defaults by adding a check for non-null values in the HAR file during importing, esp. for cdn. Tweak expAge logic to base the expiration window on the requests start time (rather than $now).

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1064 fc7d47d3-c008-acd5-f51f-d19787b8a02f

3725b5ef265269cf5c00d27dc05be560f95c10c3 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
remove some comments

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1063 fc7d47d3-c008-acd5-f51f-d19787b8a02f

9f0223574c8e08b6804221605c07ad6932c420d5 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
In order to make the mobile & desktop crawls use the same list of URLs, we move the copying of the "urls" table to the beginning rather than the end of the desktop crawl job

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1062 fc7d47d3-c008-acd5-f51f-d19787b8a02f

e76afd796707f890a3c54e5baba06bb282513803 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
In order to make the mobile & desktop crawls use the same list of URLs, we move the copying of the "urls" table to the beginning rather than the end of the desktop crawl job

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1061 fc7d47d3-c008-acd5-f51f-d19787b8a02f

92d0621a449d60cf39ca49be0c9eac6d712f30f1 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
Tweak the layout of the list of dump files to be a bit pithier.

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1060 fc7d47d3-c008-acd5-f51f-d19787b8a02f

34c0e06966b7a02c41644cd93bad394d9671c71d authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
remove final(?) spot where production website accesses requests table

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1059 fc7d47d3-c008-acd5-f51f-d19787b8a02f

8cc81d6565380db9f96f271fd8f8e291859c2201 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
no longer appropriate - code is out of date and not worth updating

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1058 fc7d47d3-c008-acd5-f51f-d19787b8a02f

e427f2c412da3c0fdc439ad25b17c80bc9a22200 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
So nice - finally starting to eliminate ALL dependencies on the "requests" table and also leverage the "crawls" table to make things faster. Rewrite function sliceCond() to use rank instead of a list of hardcoded URLs (altho crawls before we tracked rank still use that hardcoded list). Rewrite function getStatsDataForUrl to not use requests table and instead use the new meta-information in the pages table (eg, numHttps).

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1057 fc7d47d3-c008-acd5-f51f-d19787b8a02f

c4a31a3d3b434be4e05ba576d686791ef9d4aeb2 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
add a CVSNO comment

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1056 fc7d47d3-c008-acd5-f51f-d19787b8a02f

3782478e1a800ac25784080a349841b924f42d12 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
update the list of download files to the new set of dump files separating pages from requests and saving in mysql AND csv format.

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1055 fc7d47d3-c008-acd5-f51f-d19787b8a02f

02db6daf9d2a2972adf169419c1fbe100aca05e6 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
changes

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1054 fc7d47d3-c008-acd5-f51f-d19787b8a02f

52f5cd6091f98b688d2cc089848feabf25a92c74 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
add a CVSNO comment to use the crawls table. remove "DPRINT" from the dprint() output.

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1053 fc7d47d3-c008-acd5-f51f-d19787b8a02f

54b8e2b7b635ba773daeb05e31a31d540215a79f authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
add summary stats. adapt for mobile.

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1052 fc7d47d3-c008-acd5-f51f-d19787b8a02f

8a3ad5ab959690a195bf1ecd07dbbc5a0abed38f authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
add some adult sites. remove moot comments. add time value to all print functions.

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1051 fc7d47d3-c008-acd5-f51f-d19787b8a02f

d572e314829276be10614286fe6ffb8524a2306f authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
add patchwork.js

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1050 fc7d47d3-c008-acd5-f51f-d19787b8a02f

be51045492d2bdccf8ccfd7eb5cd4db372492ed6 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
remove unused "tafter"

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1049 fc7d47d3-c008-acd5-f51f-d19787b8a02f

72a0eec51e926dcde4def863017da309d2716ef5 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
add a comment

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1048 fc7d47d3-c008-acd5-f51f-d19787b8a02f

45b361e19c0ce31cf4267d022ac39916f65458ac authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
Add a fallback query for crawls that do not have rank.

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1047 fc7d47d3-c008-acd5-f51f-d19787b8a02f

9fbad7c99f1fe4a4829b6780d5470db978ff2e50 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
Restrict the # of images. Add querystring params for label, (set)TimeOut, and width. Add label select list. Add select list for size. Add a fallback query for crawls that do not have rank.

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1046 fc7d47d3-c008-acd5-f51f-d19787b8a02f

ab27a39b33cb2f196dabc83d8f4ef89ea0e38e20 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
nice - use patchwork.js to cut down on the # of requests

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1045 fc7d47d3-c008-acd5-f51f-d19787b8a02f

6c24814ccaceca47e1758cacaaa2a54a4deb3553 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
nice - use patchwork.js to cut down on the # of requests

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1044 fc7d47d3-c008-acd5-f51f-d19787b8a02f

20bd6c806a2c5110a4dd18eefe1bf9dfabd1627f authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
for patchwork.php

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1043 fc7d47d3-c008-acd5-f51f-d19787b8a02f

57d0c95a5e71285d6a1dbb0032c47c8c68f425b2 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
added play & pause. next adding patchwork.js.

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1042 fc7d47d3-c008-acd5-f51f-d19787b8a02f

af3ec56f2669d4e1573e5a6802c6bcb123e656fc authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
looks good - doing a commit before changing stuff

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1041 fc7d47d3-c008-acd5-f51f-d19787b8a02f

d2a59bdfd1c3fecf8c5ec7c3b2d3e3355dbc26a0 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
redirect to the appropriate filmstrip frame for a site given a time

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1040 fc7d47d3-c008-acd5-f51f-d19787b8a02f

a1f9ce5dc946dfb315b3f1f93eeebec87368f112 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
show multiple sites rendering

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1039 fc7d47d3-c008-acd5-f51f-d19787b8a02f

71b7ed042d400977f1130a641f2d4779a394f2ac authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
Add new stats for reqGif, reqJpg, reqPng, bytesGif, bytesJpg, bytesPng, reqFont, bytesFont, . Calculate stats (mostly in computeOther) from the pages table so we do NOT need to use the requests table any longer. Add an optional $link param to the DB access functions to reuse a connection. Tweaks to DB schema.

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1038 fc7d47d3-c008-acd5-f51f-d19787b8a02f

a996e3fceead79bc1cb2812a4b0118926ebae6c4 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
add new function resourcesAvailableFromTable()

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1037 fc7d47d3-c008-acd5-f51f-d19787b8a02f

44763fe7517aa103360c15adc59cf467cfde5ec5 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
make the dump file regex more flexible to accomodate the new pages & requests & CSV dumps

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1036 fc7d47d3-c008-acd5-f51f-d19787b8a02f

46424e7c4eeb45e0d03ffb4751224ac669863650 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
many changes. currently using this version to update Jul 1 2012 thru Oct 15 2012.

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1035 fc7d47d3-c008-acd5-f51f-d19787b8a02f

69d4204976088e9cc5fe04ee15c1ab1c360b6eb0 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
Near final version of script with some profiling code trying to figure out why the updates lock.

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1034 fc7d47d3-c008-acd5-f51f-d19787b8a02f

240ac74ea33b9d33f5e7aa9e2ba5e825eddc2752 authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
script for updating the DB after the major schema changes in Dec 2012

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1033 fc7d47d3-c008-acd5-f51f-d19787b8a02f

2e30bdbb0b7118aaf225add6940b177a65bceefa authored almost 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
do NOT replace existing records in the pages table - just keep the initial one

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1032 fc7d47d3-c008-acd5-f51f-d19787b8a02f

3f440a651e9b405f864cccfcd5337d7e6dd9d962 authored about 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
Now that parsing is 100x faster, reduce the number of processes from 10 to 5. Do better handling of parameters in obtainTestsWithCode (to avoid issues with modulo that are similar to what was just fixed). Major change that made parse 100x faster - avoid doing REPLACE and instead do INSERT. Also eliminate SELECT statement by using LAST_INSERT_ID().

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1031 fc7d47d3-c008-acd5-f51f-d19787b8a02f

40f8724161f085700818eb5407dafbaea6655e51 authored about 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
dump the crawls table sql and data

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1030 fc7d47d3-c008-acd5-f51f-d19787b8a02f

719650054882d32ccdab74a5b2a08c167c32397c authored about 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
rewrite downloads to include crawls table and better instructions

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1029 fc7d47d3-c008-acd5-f51f-d19787b8a02f

a4a6b60501df2e178ab8242902619683c5f41ad4 authored about 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
MAJOR BUG FIX: I was doing the modulo wrong - it was NOT zero-based before. This caused an increase in duplicate records (because 2 processes would be inserting the same HAR file). Duplicate records required doing a REPLACE which would delete a row which would block all other writes in the other processes. Fixing this bug made the parse code 100x faster!

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1028 fc7d47d3-c008-acd5-f51f-d19787b8a02f

d66789d8f2ac1d636a604164ad09da8899ce35fe authored about 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
add $gCrawlsTable. add doLastInsertId function.

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1027 fc7d47d3-c008-acd5-f51f-d19787b8a02f

52cd7dc6b6b66b845a42470e0d89c2728cc6709b authored about 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
add semi-colons

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1026 fc7d47d3-c008-acd5-f51f-d19787b8a02f

8aacff066b7e62551e29ba02bdc5372ff198b6af authored about 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
create apple icons

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1025 fc7d47d3-c008-acd5-f51f-d19787b8a02f

67d9bf5ce82c5503015d69c7625ea1bed65a64f2 authored about 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
push apple icons

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1024 fc7d47d3-c008-acd5-f51f-d19787b8a02f

e5f4bc31bfc72c179b3455c8358abb5c9f23bd5f authored about 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
bail if there are no runs currently in the status table

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1023 fc7d47d3-c008-acd5-f51f-d19787b8a02f

e033eea0fc11b429e00b6dc6f296fea2ed0b6071 authored about 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
Fix bugs in new fields: use correct header DB names. harvest data for bytesHtmlDoc, numRedirects, numErrors, numGlibs, numHttps, numCompressed, & maxDomainReqs. change reqfont to reqFont. fix preg_match pattern.

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1022 fc7d47d3-c008-acd5-f51f-d19787b8a02f

6ddb6b857387b201e2ec7938d73c977a3f2d7f46 authored about 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
aggregate maxage status for each page

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1021 fc7d47d3-c008-acd5-f51f-d19787b8a02f

74b327ccdeda5f3949aca731221ad100aeaf1ad3 authored about 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
add expAge to "requests" table

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1020 fc7d47d3-c008-acd5-f51f-d19787b8a02f

45b5a6ef8ae316ea7e908c1be7cc2a92d00dc7ff authored about 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
calculate expAge

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1019 fc7d47d3-c008-acd5-f51f-d19787b8a02f

e5359fba7608d031407e3a2ba3972aa2cab84d0f authored about 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
BIG CHANGES: Rearrange where we get a lot of the "meta" data - instead of pulling from xmlresults get it from the HAR file (like startRender, pagespeedScore, loadTime, domElements). Get new data: fullyLoaded, visualComplete, gzipTotal, gzipSavings, SpeedIndex. Gather font request info.

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1018 fc7d47d3-c008-acd5-f51f-d19787b8a02f

da6802d9126924f23c337dfb2d87e839c9f0757c authored about 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
remove "archive" when selecting crawls. Add columns: cdn, fullyLoaded, visualComplete, gzipTotal, & gzipSavings.

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1017 fc7d47d3-c008-acd5-f51f-d19787b8a02f

7206b78f8d3faa3487baf806939963501a81f2ac authored about 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
add comments to columns in "pages" table

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1016 fc7d47d3-c008-acd5-f51f-d19787b8a02f

16a62cf531d15a50cc35f9cca16fbbdc7f02f428 authored about 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
MAJOR changes to "pages" table adding/removing columns in preparation for removing "requests" table from production and easier calculation of aggregate stats.

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1015 fc7d47d3-c008-acd5-f51f-d19787b8a02f

4815368551bf36406a6d3ca2fa57a5e9b130bf21 authored about 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
Note the change from IE8 to IE9. Fix links to Blaze.io and add Akamai.

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1014 fc7d47d3-c008-acd5-f51f-d19787b8a02f

145edb874539451baaff357a32e00e9a7795080e authored about 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
No longer save urlHtml to pages table. Round the batch report % of errors.

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1013 fc7d47d3-c008-acd5-f51f-d19787b8a02f

aa85f54ca6f5339af641125cce41ec0a7fc5fa1d authored about 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
mark function deprecated

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1012 fc7d47d3-c008-acd5-f51f-d19787b8a02f

7ec0a4c377951536f2d97eefa633ef090a988017 authored about 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
avoid error where we retrieve a label that is not yet done.

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1011 fc7d47d3-c008-acd5-f51f-d19787b8a02f

1930a8d6b9bc86b842f251df9ce493cfb1674944 authored about 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
remove dependency on "requests" table - instead get the requests info from the HAR file on WPT.

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1010 fc7d47d3-c008-acd5-f51f-d19787b8a02f

1e5dfb7ae945298f7f4587f7e7b00aecae52c301 authored about 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
use "resource" instead of "request".

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1009 fc7d47d3-c008-acd5-f51f-d19787b8a02f

5319e76565423a630a49df4d68b18627220baebd authored about 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
wow - there was code in here to download AN ENTIRE CRAWL as CSV. Removed that code. Tweaked function tdstat() to just return the value (do not prettify).

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1008 fc7d47d3-c008-acd5-f51f-d19787b8a02f

16e0ef78885a965b5e8263ddc2dbefa4769cace1 authored about 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
new more object-oriented organization of code

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1007 fc7d47d3-c008-acd5-f51f-d19787b8a02f

c34b13789edafa426114b4464e3f28027c8e8938 authored about 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
remove dependency on the "requests" table. Instead, re-fetch and re-parse the HAR file.

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1006 fc7d47d3-c008-acd5-f51f-d19787b8a02f

f4275e30ff17ae1b2245b398c567071401a0f072 authored about 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
no longer use $ghReqOtherHeaders and $ghRespOtherHeaders. Fix bug where we were truncating cookie values BEFORE measuring the cookielen. Similar bug for saving "other" headers. use $origValue instead. no longer use redirectUrlShort and urlHtmlShort for requests.

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1005 fc7d47d3-c008-acd5-f51f-d19787b8a02f

c7cf076406e4922e599bc7a57bf1ae0e91977ad3 authored about 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
no longer use $ghReqOtherHeaders and $ghRespOtherHeaders

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1004 fc7d47d3-c008-acd5-f51f-d19787b8a02f

58fbad99c364346919f60713fa4826e148a25988 authored about 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
rename $row to either $pageData or $req. no logic changes.

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1003 fc7d47d3-c008-acd5-f51f-d19787b8a02f

1902222ac43bff3bd89863857009405399c9c663 authored about 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>
remove $gaDevices and deviceNames() - not used anywhere. add curDevice() and isMobile(). optimize latestLabel() to use crawls table - now it takes 0 seconds instead of 8!

git-svn-id: http://httparchive.googlecode.com/svn/trunk@1002 fc7d47d3-c008-acd5-f51f-d19787b8a02f

ce7fc40db93e7de5f7091a506fbc1b9ebfb9c13a authored about 12 years ago by [email protected] <[email protected]@fc7d47d3-c008-acd5-f51f-d19787b8a02f>