From 3b7e3526159cd2122d6c752778e4fe112c28360e Mon Sep 17 00:00:00 2001 From: Chris Ritzo Date: Thu, 16 Apr 2020 10:22:31 -0400 Subject: [PATCH 1/2] added example from issue6 --- ...ate_range-matching-ndt5-tcpinfo-on-UUID.md | 68 +++++++++++++++++++ 1 file changed, 68 insertions(+) create mode 100644 examples/NDT-US-state-date_range-matching-ndt5-tcpinfo-on-UUID.md diff --git a/examples/NDT-US-state-date_range-matching-ndt5-tcpinfo-on-UUID.md b/examples/NDT-US-state-date_range-matching-ndt5-tcpinfo-on-UUID.md new file mode 100644 index 0000000..ce0d00b --- /dev/null +++ b/examples/NDT-US-state-date_range-matching-ndt5-tcpinfo-on-UUID.md @@ -0,0 +1,68 @@ +# NDT Upload and Download Tests from a US State within a Defined Date Range, with Fields from ndt5 and tcpinfo tables, matching on test UUID + +Tags: +- unified views +- geography +- UUID matching +- ndt5 +- tcpinfo + +The queries below return upload and download test results from a specified US State, within a defined date range. They use the NDT "Helpful Views", _measurement-lab.ndt.unified_uploads_ and _measurement-lab.ndt.unified_downloads_, and also bring in specific fields from the `ndt5` and `tcpinfo` tables by matching on each test's Universally Unique Identifier, or UUID. + +## Upload Query +```~sql +#standardSQL +SELECT + tcpinfo.ParseInfo.TaskFileName AS test_id, + a.TestTime AS UTC_date_time, + ndt.client.IP AS client_ip, + ndt.Client.Geo.latitude AS client_latitude, + ndt.Client.Geo.longitude AS client_longitude, + ndt.Client.Geo.country_code AS country_code, + ndt.Client.Geo.region AS region, + ndt.Client.Geo.city AS city, + ndt.Client.Geo.postal_code AS postal_code, + a.MeanThroughputMbps AS uploadThroughput, + NULL AS downloadThroughput, + TIMESTAMP_DIFF(ndt5.result.S2C.EndTime, ndt5.result.S2C.StartTime, MICROSECOND) AS duration, + FinalSnapshot.TCPInfo.BytesReceived AS HCThruOctetsRecv +FROM `measurement-lab.ndt.unified_uploads` ndt, + `measurement-lab.ndt.ndt5` ndt5, + `measurement-lab.ndt.tcpinfo` tcpinfo +WHERE + a.UUID = tcpinfo.UUID + AND a.UUID = ndt5.result.S2C.UUID + AND test_date BETWEEN '2020-01-01' AND '2020-03-31' + AND ndt.Client.Geo.country_code = 'US' + AND ndt.Client.Geo.region = 'MD' +ORDER BY UTC_date_time ASC +``` + +## Download Query +```~sql +#standardSQL +SELECT + tcpinfo.ParseInfo.TaskFileName AS test_id, + a.TestTime AS UTC_date_time, + ndt.client.IP AS client_ip, + ndt.Client.Geo.latitude AS client_latitude, + ndt.Client.Geo.longitude AS client_longitude, + ndt.Client.Geo.country_code AS country_code, + ndt.Client.Geo.region AS region, + ndt.Client.Geo.city AS city, + ndt.Client.Geo.postal_code AS postal_code, + a.MeanThroughputMbps AS downloadThroughput, + NULL AS uploadThroughput, + TIMESTAMP_DIFF(ndt5.result.S2C.EndTime, ndt5.result.S2C.StartTime, MICROSECOND) AS duration, + FinalSnapshot.TCPInfo.BytesReceived AS HCThruOctetsRecv +FROM `measurement-lab.ndt.unified_downloads` ndt, + `measurement-lab.ndt.ndt5` ndt5, + `measurement-lab.ndt.tcpinfo` tcpinfo +WHERE + a.UUID = tcpinfo.UUID + AND a.UUID = ndt5.result.S2C.UUID + AND test_date BETWEEN '2020-01-01' AND '2020-03-31' + AND ndt.Client.Geo.country_code = 'US' + AND ndt.Client.Geo.region = 'MD' +ORDER BY UTC_date_time ASC +``` From a4453d409641af68e4a5f6097f75a277fc96d692 Mon Sep 17 00:00:00 2001 From: Chris Ritzo Date: Thu, 16 Apr 2020 10:39:10 -0400 Subject: [PATCH 2/2] updated README.md to include link to examples folder --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 92c9392..34af0b0 100644 --- a/README.md +++ b/README.md @@ -12,4 +12,4 @@ Please fork this repository and use the examples for your own learning. The M-La The examples and code included in this repository are organized according to the tool used. For example, to learn about using [R](https://www.r-project.org/) / [R Studio](https://www.rstudio.com/) to query, analyze, and visualize M-Lab data, look in the [**R**](https://github.com/m-lab/data-support/tree/master/R) folder. -General resources for getting started with M-Lab data can be found in our [Data Documentation](https://www.measurementlab.net/data/docs/) on the M-Lab website. \ No newline at end of file +General examples are currently being organized in the folder, [examples](examples/).