From 3abf5cd2f37735a43c7411100c2cf0bbcbe23bfe Mon Sep 17 00:00:00 2001 From: David Schoonover Date: Sun, 3 Jun 2012 16:33:47 +0200 Subject: [PATCH] Adds graph spec; some housekeeping. --- docs/internals/d3-mvc.md | 29 ---------------- docs/internals/dataset-spec.yaml | 48 --------------------------- docs/internals/housekeeping.md | 56 -------------------------------- docs/internals/specs/dataset-spec.yaml | 48 +++++++++++++++++++++++++++ docs/internals/specs/graph-spec.yaml | 52 +++++++++++++++++++++++++++++ docs/internals/vis-editor.md | 2 - docs/notes.md | 52 ----------------------------- docs/notes/d3-mvc.md | 29 ++++++++++++++++ docs/notes/housekeeping.md | 56 ++++++++++++++++++++++++++++++++ docs/notes/notes.md | 52 +++++++++++++++++++++++++++++ docs/notes/option-tag-notes.md | 24 +++++++++++++ docs/notes/tag-notes.md | 24 ------------- 12 files changed, 261 insertions(+), 211 deletions(-) delete mode 100644 docs/internals/d3-mvc.md delete mode 100644 docs/internals/dataset-spec.yaml delete mode 100644 docs/internals/housekeeping.md create mode 100644 docs/internals/specs/dataset-spec.yaml create mode 100644 docs/internals/specs/graph-spec.yaml delete mode 100644 docs/internals/vis-editor.md delete mode 100644 docs/notes.md create mode 100644 docs/notes/d3-mvc.md create mode 100644 docs/notes/housekeeping.md create mode 100644 docs/notes/notes.md create mode 100644 docs/notes/option-tag-notes.md delete mode 100644 docs/notes/tag-notes.md diff --git a/docs/internals/d3-mvc.md b/docs/internals/d3-mvc.md deleted file mode 100644 index f83f243..0000000 --- a/docs/internals/d3-mvc.md +++ /dev/null @@ -1,29 +0,0 @@ -# d3.mvc - -An MVC framework built on top of `d3`. - - -## d3.model - -d3.model(attrs) -> d3.Model - -### Attribute Accessors - -- attr() -> Map -- attr(k) -> * -- attr(k, v) -> this -- has(k) -> Boolean -- get(k, [default]) -> * -- set(k, v) -> this -- del(k) -> this - - - - - -## d3.view - - - - - diff --git a/docs/internals/dataset-spec.yaml b/docs/internals/dataset-spec.yaml deleted file mode 100644 index 7679ca0..0000000 --- a/docs/internals/dataset-spec.yaml +++ /dev/null @@ -1,48 +0,0 @@ -### Kraken Datasource Spec ### - -# Describes the format of datasource json/yaml files for use in the Kraken chart UI. -# Reminder: the project transparently compiles any .yaml files requested as .json -# at request time -- you're welcome to generate them in either form. - -# TODO: Thrift/Avro schema - - -id : Unique identifier for this datasource. This file should be named `{id}.{ext}` - and live in the data directory alongside the data file. -name : Human-friendly name for the datasource. -shortName : A shorter version of the datasource name for use in the UI. -title : Title for use on the graph of the data in some chart-types. -subtitle : A subtitle for use on the graph of the data in some chart-types. -desc : Description of the datasource. -notes : Notes about the datasource, particularly about definitions, data integrity, - data normalization, data generation, etc. - -format : Data format. Valid: csv, json. (Maybe in the future: yaml, xml, etc.) -url : Canonical URL for the datasource's data file, eg: /data/datasources/{id}.csv - - -### Timespan and quanta covered by this datasource -timespan : - start : Start of the data, eg: 2012/01/01T00:00:00 or 2012/01/01 or 1325404800 - All dates should be seconds-since-the-epoch, or ISO-8601 formatted: YYYY/MM/DDThh:mm:ss.s - See also: http://www.w3.org/TR/NOTE-datetime - And: http://blog.dygraphs.com/2012/03/javascript-and-dates-what-mess.html - end : End of the data, eg: 2012/02/01T00:00:00 or 2012/02/01 or 1328083200 - step : Amount of time between elapsed between each row, measured in seconds, eg: 86400 - -### Metadata about the Datatypes -# Note: 'columns' can also be supplied as a list of { label, type, id } objects instead of two lists. (TODO) -columns : - ids : List of the column-ids in order. This is like column-label, except it must be unique - for the dataset. This allows charts to specify their metrics by (dataset, col_id) - rather than column-index, making the chart independent of column-order. - labels : List of the column-names in order. (Future optimization: the date column could - be omitted if the data is sorted and contains no gaps -- each datapoint is exactly - timespan.step apart, and none are missing.) - types : List of the column datatype-names in order. Valid type names: date, float, int. - -### Defaults for charting -chart : - chartType : Visualization library to use. Valid: dygraphs. - options : Dict of option-names to values. - diff --git a/docs/internals/housekeeping.md b/docs/internals/housekeeping.md deleted file mode 100644 index c464fd2..0000000 --- a/docs/internals/housekeeping.md +++ /dev/null @@ -1,56 +0,0 @@ -# Housekeeping - -## Base -- BaseModel - - `.fromURL()` -- use `.chartType.parseValue k, v` for parsers -- BaseView - - `.modelEvents` or maybe `.events.{ el, model, view... }` -- AppView for common `main()` tasks -- ScaffoldView: generic form for a model's fields - - Bootstrap horizontal-form - - Hooks for filtering visibility - - Controls: Save, Load, Cancel; Delete - - -## Edit UI -- Backbone.Router? -- Loading Indicator: - - Render UI at start with `loading` mode, showing placeholders and indicator - - Activate Loading Spinner post-ready when waiting - - -## Data UI -- Timespan compat check -- Metric: `column_idx` -> `column_id` -- Tab-pane height fix? all `inline-block`: ` ` -- Should data files have timespan in name? - - -## Graph Options UI -- `tags.json` to describe colors, sorting, ignored-tags -- Outline modified -- Fix UI pill color when expanded -- Filter: - - by major Categories: labels, grid, axes, display, interactivity - - by Tag (collapsable box) - - via typeahead box: names + tags -- Sort - - by: Modified, Importance, Tag, Lexical - - Use `categoryRows` -- Derived Options: - - visibility - - colors - - dateWindow - - file - - width, height -- Buttons: Reset {all,visible} to defaults - - -## Misc -- Generate markdoc wiki from `/docs` on Build, Deploy -- Refactor directories: - - dataset/ - dataset/ +metric - datasource/ +column -- Wrap `Backbone.extend()` to fire `subclass` event on parent class - diff --git a/docs/internals/specs/dataset-spec.yaml b/docs/internals/specs/dataset-spec.yaml new file mode 100644 index 0000000..47fc0b7 --- /dev/null +++ b/docs/internals/specs/dataset-spec.yaml @@ -0,0 +1,48 @@ +### Limn Datasource Spec ### +# +# Describes the format of datasource json/yaml files for use in the Limn chart UI. +# +# Reminder: the project transparently compiles any .yaml files requested as .json +# at request time -- you're welcome to generate them in either form. +# TODO: Thrift/Avro schema? + + +id : Unique identifier for this datasource. This file should be named `{id}.{ext}` + and live in the data directory alongside the data file. +name : Human-friendly name for the datasource. +shortName : A shorter version of the datasource name for use in the UI. +title : Title for use on the graph of the data in some chart-types. +subtitle : A subtitle for use on the graph of the data in some chart-types. +desc : Description of the datasource. +notes : Notes about the datasource, particularly about definitions, data integrity, + data normalization, data generation, etc. + +format : Data format. Valid: csv, json. (Maybe in the future: yaml, xml, etc.) +url : Canonical URL for the datasource's data file, eg: /data/datasources/{id}.csv + + +### Timespan and quanta covered by this datasource +timespan : + start : Start of the data, eg: 2012/01/01T00:00:00 or 2012/01/01 or 1325404800 + All dates should be seconds-since-the-epoch, or ISO-8601 formatted: YYYY/MM/DDThh:mm:ss.s + See also: http://www.w3.org/TR/NOTE-datetime + And: http://blog.dygraphs.com/2012/03/javascript-and-dates-what-mess.html + end : End of the data, eg: 2012/02/01T00:00:00 or 2012/02/01 or 1328083200 + step : Amount of time between elapsed between each row, measured in seconds, eg: 86400 + +### Metadata about the Datatypes +# Note: 'columns' can also be supplied as a list of { label, type, id } objects instead of two lists. (TODO) +columns : + ids : List of the column-ids in order. This is like column-label, except it must be unique + for the dataset. This allows charts to specify their metrics by (dataset, col_id) + rather than column-index, making the chart independent of column-order. + labels : List of the column-names in order. (Future optimization: the date column could + be omitted if the data is sorted and contains no gaps -- each datapoint is exactly + timespan.step apart, and none are missing.) + types : List of the column datatype-names in order. Valid type names: date, float, int. + +### Defaults for charting +chart : + chartType : Visualization library to use. Valid: dygraphs. + options : Dict of option-names to values. + diff --git a/docs/internals/specs/graph-spec.yaml b/docs/internals/specs/graph-spec.yaml new file mode 100644 index 0000000..3069622 --- /dev/null +++ b/docs/internals/specs/graph-spec.yaml @@ -0,0 +1,52 @@ +### Graph Spec ### +# +# Describes the format of graph json/yaml files saved by the Limn framework. + + +id : Unique identifier for this graph. This file will be named {id}.json + and live in the data directory alongside the data file. Defaults to the slug. +slug : Unique identifier used in URLs for the graph. + +name : Human-friendly name given to the graph. +shortName : A shorter version of the graph name for use in the UI. +desc : Description of the graph. + + +width : Width in pixels of the graph, or "auto" to fit to 100% of the container. +height : Height in pixels of the graph, or "auto" to fit to 100% of the container. +parents : Array of graph-ids from which to inherit values. + +### Metadata about the Data Sources and Metrics +data: + palette: Name of a color palette to use to generate new default colors for columns. + # Metrics is an array of Metric objects, matching the description below + metrics: + label : Human-readable label for this metric. If empty, defaults to the column + label for the datasource. + type : DataType for this column -- usually "int". + disabled : Whether this metric is disabled. Disabled metrics behave exactly like + deleted/non-existant metrics, except that they still appear in the dataset UI. + timespan : + start : Start of the data, eg: 2012/01/01T00:00:00 or 2012/01/01 or 1325404800 + All dates should be seconds-since-the-epoch, or ISO-8601 formatted: YYYY/MM/DDThh:mm:ss.s + See also: http://www.w3.org/TR/NOTE-datetime + And: http://blog.dygraphs.com/2012/03/javascript-and-dates-what-mess.html + end : End of the data, eg: 2012/02/01T00:00:00 or 2012/02/01 or 1328083200 + step : Amount of time between elapsed between each row, measured in seconds, eg: 86400 + + # DataSource + source_id : Identifier for the datasource that backs this metric. + source_col : Column in the datasource that backs this metric. + + # Chart Options + color : Color of the line in the chart. + visible : Whether this metric begins hidden in the display. + format_value : Format string or method to use to format displayed metric values. + format_axis : Format string or method to use to format displayed metric values in the axis. + + transforms : List of transforms to apply to the data before display. + scale : Scaling factor for this metric (a shortcut for transforming the data). + +### Defaults for charting +chartType : Visualization library to use. Valid: "dygraphs". +options : Dict of chart option names to values. diff --git a/docs/internals/vis-editor.md b/docs/internals/vis-editor.md deleted file mode 100644 index 5c6a335..0000000 --- a/docs/internals/vis-editor.md +++ /dev/null @@ -1,2 +0,0 @@ -# Visualization Editor - diff --git a/docs/notes.md b/docs/notes.md deleted file mode 100644 index c6967b1..0000000 --- a/docs/notes.md +++ /dev/null @@ -1,52 +0,0 @@ -# notes - -## GraphKit - - - - - -## Data UI - - -## Value Formatter - -``` -[sign][#][minW][~maxW][,][.precision][type] - -sign ::= "+" | "-" | " " -# ::= Output will be prefixed by '0b' for binary, '0o' for octal, or '0x' for hexadecimal -minW ::= Minimum field width -maxW ::= Maximum field width -, ::= Use comma for thousands separator -precision ::= How many digits displayed... - fF - after the decimal point - gG - before and after the decimal point -type ::= Use type-specific formatting: - String: s - Integer: b (base2) | c (unicode chars) | d (base10) | o (base8) | xX (base16) | n (locale-aware) - Float: eE (scientific) | fF (fixed) | gG (general) | % | - k (use KMBT when rounding; ) -``` - - -## Start Server in Screen & Tee Logs - - screen -dr kraken - ./lib/server/server.co | tee -a logs/kraken.log - - -## Deployer Config - - set -xg KRAKEN_DEPLOY_HOST dsc@reportcard2.pmtpa.wmflabs - set -xg KRAKEN_DEPLOY_PATH /srv/reportcard/kraken-ui/ - set -xg KRAKEN_DEV_HOST master.reportcard.wmflabs.org - - -## Data UI - -- Data UI = DataView - - DataSet UI = DataSetView - - DataSetMetricView - - Metric UI = MetricEditView - - DataSource UI = DataSourceUIView \ No newline at end of file diff --git a/docs/notes/d3-mvc.md b/docs/notes/d3-mvc.md new file mode 100644 index 0000000..f83f243 --- /dev/null +++ b/docs/notes/d3-mvc.md @@ -0,0 +1,29 @@ +# d3.mvc + +An MVC framework built on top of `d3`. + + +## d3.model + +d3.model(attrs) -> d3.Model + +### Attribute Accessors + +- attr() -> Map +- attr(k) -> * +- attr(k, v) -> this +- has(k) -> Boolean +- get(k, [default]) -> * +- set(k, v) -> this +- del(k) -> this + + + + + +## d3.view + + + + + diff --git a/docs/notes/housekeeping.md b/docs/notes/housekeeping.md new file mode 100644 index 0000000..c464fd2 --- /dev/null +++ b/docs/notes/housekeeping.md @@ -0,0 +1,56 @@ +# Housekeeping + +## Base +- BaseModel + - `.fromURL()` -- use `.chartType.parseValue k, v` for parsers +- BaseView + - `.modelEvents` or maybe `.events.{ el, model, view... }` +- AppView for common `main()` tasks +- ScaffoldView: generic form for a model's fields + - Bootstrap horizontal-form + - Hooks for filtering visibility + - Controls: Save, Load, Cancel; Delete + + +## Edit UI +- Backbone.Router? +- Loading Indicator: + - Render UI at start with `loading` mode, showing placeholders and indicator + - Activate Loading Spinner post-ready when waiting + + +## Data UI +- Timespan compat check +- Metric: `column_idx` -> `column_id` +- Tab-pane height fix? all `inline-block`: ` ` +- Should data files have timespan in name? + + +## Graph Options UI +- `tags.json` to describe colors, sorting, ignored-tags +- Outline modified +- Fix UI pill color when expanded +- Filter: + - by major Categories: labels, grid, axes, display, interactivity + - by Tag (collapsable box) + - via typeahead box: names + tags +- Sort + - by: Modified, Importance, Tag, Lexical + - Use `categoryRows` +- Derived Options: + - visibility + - colors + - dateWindow + - file + - width, height +- Buttons: Reset {all,visible} to defaults + + +## Misc +- Generate markdoc wiki from `/docs` on Build, Deploy +- Refactor directories: + - dataset/ + dataset/ +metric + datasource/ +column +- Wrap `Backbone.extend()` to fire `subclass` event on parent class + diff --git a/docs/notes/notes.md b/docs/notes/notes.md new file mode 100644 index 0000000..c6967b1 --- /dev/null +++ b/docs/notes/notes.md @@ -0,0 +1,52 @@ +# notes + +## GraphKit + + + + + +## Data UI + + +## Value Formatter + +``` +[sign][#][minW][~maxW][,][.precision][type] + +sign ::= "+" | "-" | " " +# ::= Output will be prefixed by '0b' for binary, '0o' for octal, or '0x' for hexadecimal +minW ::= Minimum field width +maxW ::= Maximum field width +, ::= Use comma for thousands separator +precision ::= How many digits displayed... + fF - after the decimal point + gG - before and after the decimal point +type ::= Use type-specific formatting: + String: s + Integer: b (base2) | c (unicode chars) | d (base10) | o (base8) | xX (base16) | n (locale-aware) + Float: eE (scientific) | fF (fixed) | gG (general) | % | + k (use KMBT when rounding; ) +``` + + +## Start Server in Screen & Tee Logs + + screen -dr kraken + ./lib/server/server.co | tee -a logs/kraken.log + + +## Deployer Config + + set -xg KRAKEN_DEPLOY_HOST dsc@reportcard2.pmtpa.wmflabs + set -xg KRAKEN_DEPLOY_PATH /srv/reportcard/kraken-ui/ + set -xg KRAKEN_DEV_HOST master.reportcard.wmflabs.org + + +## Data UI + +- Data UI = DataView + - DataSet UI = DataSetView + - DataSetMetricView + - Metric UI = MetricEditView + - DataSource UI = DataSourceUIView \ No newline at end of file diff --git a/docs/notes/option-tag-notes.md b/docs/notes/option-tag-notes.md new file mode 100644 index 0000000..afb83cf --- /dev/null +++ b/docs/notes/option-tag-notes.md @@ -0,0 +1,24 @@ + +chart: title, titleHeight, + +axis-appearance + +x-axis: xlabel, xLabelHeight, drawXAxis, xAxisHeight, xAxisLabelWidth, xValueParser, drawXGrid, + +y-axis: ylabel, y2label, yLabelHeight, avoidMinZero, drawYAxis, includeZero, logscale, yAxisLabelWidth, valueRange, drawYGrid, labelsKMB, digitsAfterDecimal, labelsKMG2, maxNumberWidth, sigFigs, + +axes: axis, axisLabelColor, axisLabelFontSize, axisLabelFormatter, axisLabelWidith, axisLineColor, axisLineWidth, axisTickSize, pixelPerLabel, ticker, dateWindow + +line: connectSeparatedPoints, drawPoints, pointSize, strokePattern, strokeWidth, colorSaturation, colorValue, colors, + +graph: fillGraph, stackedGraph, stepPlot, panEdgeFraction, gridLineColor, gridLineWidth, highlightCircleSize, rangeSelectorHeight, rangeSelectorPlotFillColor, rangeSelectorPlotStrokeColor, showRangeSelector, rightGap, timingName, visibility + +legend: labelsDiv, labelsDivStyles, labelsDivWidth, labelsSeparateLines, labelsShowZeroValues, legend, showLabelsOnHighlight, + +callback: pointClickCallback, annotationClickHandler, annotationMouseOutHandler, annotationMouseOverHandler, valueFormatter, clickCallback, drawCallback, highlightCallback, underlayCallback, unhighlightCallback, zoomCallback, + +statistics: sigma, wilsonInterval, fillAlpha, customBars, errorBars, fractions, rollPeriod, showRoller, + +data: delimiter, displayAnnotations, + +interactivity: animatedZooms, hideOverlayOnMouseOut, isZoomedIgnoreProgrammaticZoom diff --git a/docs/notes/tag-notes.md b/docs/notes/tag-notes.md deleted file mode 100644 index afb83cf..0000000 --- a/docs/notes/tag-notes.md +++ /dev/null @@ -1,24 +0,0 @@ - -chart: title, titleHeight, - -axis-appearance - -x-axis: xlabel, xLabelHeight, drawXAxis, xAxisHeight, xAxisLabelWidth, xValueParser, drawXGrid, - -y-axis: ylabel, y2label, yLabelHeight, avoidMinZero, drawYAxis, includeZero, logscale, yAxisLabelWidth, valueRange, drawYGrid, labelsKMB, digitsAfterDecimal, labelsKMG2, maxNumberWidth, sigFigs, - -axes: axis, axisLabelColor, axisLabelFontSize, axisLabelFormatter, axisLabelWidith, axisLineColor, axisLineWidth, axisTickSize, pixelPerLabel, ticker, dateWindow - -line: connectSeparatedPoints, drawPoints, pointSize, strokePattern, strokeWidth, colorSaturation, colorValue, colors, - -graph: fillGraph, stackedGraph, stepPlot, panEdgeFraction, gridLineColor, gridLineWidth, highlightCircleSize, rangeSelectorHeight, rangeSelectorPlotFillColor, rangeSelectorPlotStrokeColor, showRangeSelector, rightGap, timingName, visibility - -legend: labelsDiv, labelsDivStyles, labelsDivWidth, labelsSeparateLines, labelsShowZeroValues, legend, showLabelsOnHighlight, - -callback: pointClickCallback, annotationClickHandler, annotationMouseOutHandler, annotationMouseOverHandler, valueFormatter, clickCallback, drawCallback, highlightCallback, underlayCallback, unhighlightCallback, zoomCallback, - -statistics: sigma, wilsonInterval, fillAlpha, customBars, errorBars, fractions, rollPeriod, showRoller, - -data: delimiter, displayAnnotations, - -interactivity: animatedZooms, hideOverlayOnMouseOut, isZoomedIgnoreProgrammaticZoom -- 1.7.0.4