Interface Matrix2D

  • All Superinterfaces:
    Iterable<DMUtil.FieldValues>, Serializable

    public interface Matrix2D
    extends Serializable, Iterable<DMUtil.FieldValues>
    Provides access to a QueryResult in a formula context, in the form of a two-dimension matrix. The matrix columns correspond to query projections, matrix rows correspond to the rows in the query result set, in the same order. Row and column indexes are zero-based. A query result may be obtained by

    • executing a Query in the DatamartContext
    • executing a Query in the TableContext
    • calling the DatamartQuery function, or equivalent SandboxAPI api.datamartQuery() - both deprecated!

    • Method Detail

      • getRowCount

        long getRowCount()
        Returns:
        Number of rows in the matrix
      • getColumnCount

        long getColumnCount()
        Returns:
        Number of columns in the matrix
      • getColumnLabel

        String getColumnLabel​(long col)
        Parameters:
        col - Index of the column to get the label for. The label is the corresponding projection's alias.
        Returns:
        The column label
      • isEmpty

        boolean isEmpty()
        A matrix is empty if it has no rows.
        Returns:
      • getColumnLabels

        List<String> getColumnLabels()
        Returns:
        The column labels from left to right.
      • getColumn

        long getColumn​(String label)
        Parameters:
        label - Label of a column in the matrix
        Returns:
        The index of that column
      • getColumns

        long[] getColumns​(Collection<String> labels)
        Parameters:
        labels - Labels of a set of columns in the matrix
        Returns:
        The indexes of those columns, in the same order
      • getColumnDataType

        DataType getColumnDataType​(String colLabel)
        Parameters:
        colLabel - Label of the column to get the DataType for
        Returns:
        DataType of the given column; null if matrix does not have any meta defined.
      • selectRow

        Matrix2D selectRow​(long row)
        Selects one row in the matrix, and returns the result as new Matrix2D object, but which crucially does not duplicate any of the original rows. Another way to look at this, is to see the result sub-matrix as a view on the original one. Any changes to the row data in one matrix, will be reflected in the other.
        Parameters:
        row - Index of row to select
        Returns:
        Sub-matrix containing that one row
      • selectRows

        Matrix2D selectRows​(long startRow,
                            long endRow)
        Selects a subset of rows in the matrix, and returns the result as new Matrix2D object, but which crucially does not duplicate any of the original rows. Another way to look at this, is to see the result sub-matrix as a view on the original one. Any changes to the row data in one matrix, will be reflected in the other.
        Parameters:
        startRow - Index of first row to select, inclusive
        endRow - Index of last row to select, inclusive
        Returns:
        Sub-matrix containing all rows from the start to end row
      • selectColumn

        Matrix2D selectColumn​(String label)
        Selects a column in the matrix, and returns the result as new Matrix2D object, but which crucially does not duplicate any of the original rows. Another way to look at this, is to see the result sub-matrix as a view on the original one. Any changes to the row data in one matrix, will be reflected in the other.
        Parameters:
        label - Label of the column to select
        Returns:
        Sub-matrix containing that one column
      • selectColumn

        Matrix2D selectColumn​(long col)
        Selects a column in the matrix, and returns the result as new Matrix2D object, but which crucially does not duplicate any of the original rows. Another way to look at this, is to see the result sub-matrix as a view on the original one. Any changes to the row data in one matrix, will be reflected in the other.
        Parameters:
        label - Index of the column to select
        Returns:
        Sub-matrix containing that one column
      • selectColumns

        Matrix2D selectColumns​(long startCol,
                               long endCol)
        Selects a subset of columns in the matrix, and returns the result as new Matrix2D object, but which crucially does not duplicate any of the original rows. Another way to look at this, is to see the result sub-matrix as a view on the original one. Any changes to the row data in one matrix, will be reflected in the other.
        Parameters:
        startCol - Index of first column to select, inclusive
        endCol - Index of last column to select, inclusive
        Returns:
        Sub-matrix containing all columns from the start to end column
      • select

        Matrix2D select​(long startRow,
                        long endRow,
                        long startColumn,
                        long endColumn)
        Selects a subset of rows and columns in the matrix, and returns the result as new Matrix2D object, but which crucially does not duplicate any of the original rows. Another way to look at this, is to see the result sub-matrix as a view on the original one. Any changes to the row data in one matrix, will be reflected in the other.
        Parameters:
        startRow - Index of first row to select, inclusive
        endRow - Index of last row to select, inclusive
        startCol - Index of first column to select, inclusive
        endCol - Index of last column to select, inclusive
        Returns:
        Sub-matrix containing all cells from the rectangular area defined by the coordinates (startRow,startCol) to (endRow,endCol)
      • selectRows

        Matrix2D selectRows​(long[] rows)
        Selects a subset of rows in the matrix, and returns the result as new Matrix2D object, but which crucially does not duplicate any of the original rows. Another way to look at this, is to see the result sub-matrix as a view on the original one. Any changes to the row data in one matrix, will be reflected in the other.
        Parameters:
        rows - Array of row indices to select.
        Returns:
        Sub-matrix containing the rows matching those indices, preserving the order from the original matrix
      • selectRows

        Matrix2D selectRows​(Collection<? extends Number> rows)
        Selects a subset of rows in the matrix, and returns the result as new Matrix2D object, but which crucially does not duplicate any of the original rows. Another way to look at this, is to see the result sub-matrix as a view on the original one. Any changes to the row data in one matrix, will be reflected in the other.
        Parameters:
        rows - Collection of row indices to select.
        Returns:
        Sub-matrix containing the rows matching those indices, preserving the order from the original matrix
      • search

        long search​(BigDecimal value,
                    long col)
        Finds the index of row containing a given value in a given column, assuming this column is sorted high --> low. if there is no exact match we pick the previous or next one, whichever is closest to the given threshold value/
        Parameters:
        value - Threshold value
        col - Index of the column to search
        Returns:
        Index of the row closest to the threshold value
      • search

        long[] search​(Collection<?> values,
                      long col)
        Find all rows for which the value in the given column is contained in the passed in collection. This would typically be done only on columns containing discrete data.
        Parameters:
        values - Collection of values to match to
        col - Index of the column to search
        Returns:
        Array of indices of matching rows
      • max

        Matrix2D max()
        Returns:
        A new one row matrix, containing the maximum (of the population of values) for each numeric column in this matrix, at the same column indices. Non-numeric columns result in NaN.
      • min

        Matrix2D min()
        Returns:
        A new one row matrix, containing the minimum (of the population of values) for of each numeric column in this matrix, at the same column indices. Non-numeric columns result in NaN.
      • sum

        Matrix2D sum()
        Returns:
        A new one row matrix, containing the sum of the values for each numeric column in this matrix, at the same column indices. Non-numeric columns result in NaN.
      • mean

        Matrix2D mean()
        Returns:
        A new one row matrix, containing the mean (of the population of values) for each numeric column in this matrix, at the same column indices. Non-numeric columns result in NaN.
      • std

        Matrix2D std()
        Returns:
        A new one row matrix, containing the standard deviation (of the population of values) for each numeric column in this matrix, at the same column indices. Non-numeric columns result in NaN.
      • indexOfMin

        Matrix2D indexOfMin()
        Returns:
        A new one row matrix, containing the index of the row that holds the minimum value, for each numeric column in this matrix, at the same column indices. Non-numeric columns result in NaN.
      • indexOfMax

        Matrix2D indexOfMax()
        Returns:
        A new one row matrix, containing the index of the row that holds the maximum value, for each numeric column in this matrix, at the same column indices. Non-numeric columns result in NaN.
      • normalize

        Matrix2D normalize​(long... cols)
        Normalizes numeric column values, by dividing each value in the column by the total for the column. If the total = 0, the normalized value is also 0.
        Parameters:
        cols - The indices of the columns to normalize. These columns should contain numerical data, for th enormalization to be useful.
        Returns:
        A reference to this Matrix2D, in which the values have been normalized 'in situ'.
      • getValue

        Object getValue()
        Convenience getter of values in the matrix.
        Returns:

        • a single object value if the matrix has just one cell
        • a map if the matrix as just one row
        • a list of object values if the matrix has just one column
        • a list of maps if the matrix has multiple rows and columns
        • null for a 0-dim matrix (with 0 columns and or 0 rows)
        • else just the matrix itself

      • getValue

        Object getValue​(long row,
                        long col)
        Parameters:
        row - Row index
        col - Column index
        Returns:
        Value of cell at coordinates (row,col)
      • getRowValues

        DMUtil.FieldValues getRowValues​(long row)
        Parameters:
        row - Row index
        Returns:
        Values on that row, in FieldValues (Map) format, i.e. [colLabel = value,...]
      • getColumnValues

        List<Object> getColumnValues​(long col)
        Parameters:
        col - Column index
        Returns:
        The values in that column in List format, in the same order.
      • setValue

        void setValue​(Object val,
                      long row,
                      long col)
        Modifies the value at the given coordinates (row,col)
        Parameters:
        val - Value to set
        row - Row index
        col - Column index
      • setRowValues

        void setRowValues​(Map<String,​Object> val,
                          long row)
        Modifies the value at the row coordinate, setting those values that match column names in the matrix.
        Parameters:
        val - Map of (column name, value) pairs.
        row - Row index
      • getPercentileValue

        BigDecimal getPercentileValue​(long col,
                                      int percentile)
        Calculates the value at the given percentile in the given numeric column.
        Parameters:
        col - Column index
        percentile - Integer number from 0 to 100
        Returns:
        The value so that percentile% of values (samples) in the column are smaller or equal to this value
      • getCumulativeDistributionValue

        Long getCumulativeDistributionValue​(long col,
                                            BigDecimal value)
        Determines the total number of values (samples) smaller or equal to the given value, in the given numeric column.
        Parameters:
        col - Column index
        value - Input to the CDF ('cumulative distribution function')
        Returns:
        CDF evaluated at that value
      • getExcelPercentileValues

        BigDecimal[] getExcelPercentileValues​(long c)
        Computes percentiles compatible with Excel
        Parameters:
        c - Column index
        Returns:
        0 to 100th percentile values in a 101-sized BigDecimal array (i.e. result[75] fives the 75th percentile).
      • getExcelPercentRank

        double getExcelPercentRank​(long c,
                                   double value)
        Computes the rank of a value, as a percentage, in the distribution of values in the given column. The values in the column do not need to be sorted upfront. The algorithm uses interpolation when there is no exact match with the passed in value.
        Parameters:
        col - Column index
        value - The value for which to return the rank
        Returns:
        The rank, from 0 to 1, denoting the percentile of the distribution in which the given value falls.
      • normalizeRowValues

        Matrix2D normalizeRowValues​(Collection<String> colLabels,
                                    int precision)
        Normalizes the values on a row-by-row basis. The value for each included value is divided by the sum of those values. If this sum is zero, the normalized values will also be zero.
        Parameters:
        colLabels - Labels of the columns to include in the normalization
        precision - Maximum number of decimals in the normalized value
        Returns:
        A reference to this Matrix2D, in which the values have been normalized 'in situ'.
      • kurtosis

        double kurtosis​(String label)
        Parameters:
        label - Label of a numeric column
        Returns:
        Kurtosis of the distribution of values in that column
      • sortRows

        Matrix2D sortRows​(long col)
        Parameters:
        col - Column index
        Returns:
        A reference to this Matrix2D, in which the rows are now ordered so that the values in the given column are sorted from low --> high
      • sortRowsReverse

        Matrix2D sortRowsReverse​(long col)
        Parameters:
        col - Column index
        Returns:
        A reference to this Matrix2D, in which the rows are now ordered so that the values in the given column are sorted from high --> low
      • getMap

        Map<String,​Object> getMap​(String keyColLabel,
                                        String valColLabel)
        Extracts a map from the values in the nominated key and value columns. The assumption is the value column is functionally dependent on the key column (as else key-->value pairs on one row may overwrite the relationship between the pair on a previous row).
        Parameters:
        keyColLabel - Label of the column for which the values are to be used as keys in the map
        valColLabel - Label of the column for which the values are to be used as values in the map
        Returns:
        Key-->value map
      • getMap

        Map<MultiKey<String>,​Object> getMap​(List<String> colLabels,
                                                  String valColLabel)
        Extracts a map from the values in the nominated key columns (resulting in keys of type MultiKey) and value columns. The assumption is the value column is functionally dependent on the key column (as else key-->value pairs on one row may overwrite the relationship between the pair on a prevoius row).
        Parameters:
        colLabels -
        valColLabel -
        Returns:
      • addMappedColumn

        Matrix2D addMappedColumn​(String colLabel,
                                 String mappedColLabel,
                                 Map<String,​Object> mapping)
        Add a column to this Matrix2D, with the value on each row looked up from the given map, using the mapped column's value as the lookup key.
        Parameters:
        colLabel - Label of the new column
        mappedColLabel - Label of the column holding the lookup keys
        mapping - Map holding the values to map those lookup keys to
        Returns:
        A reference to this matrix, to which the new column is added
      • addMappedColumn

        Matrix2D addMappedColumn​(String colLabel,
                                 List<String> fromColLabels,
                                 Map<MultiKey<String>,​Object> mapping)
        Add a column to this Matrix2D, with the value on each row looked up from the given map, using a set of existing columns' values as a composite lookup key. The keys in the map are assumed to have been instantiated by net.pricefx.formulaengine.scripting.SandboxAPI.multiKey.
        Parameters:
        colLabel - Label of the new column
        fromColLabels - Labels of the columns holding the composite lookup key values
        mapping - Map holding the values to map those lookup keys to
        Returns:
        A reference to this matrix, to which the new column is added
      • addColumn

        Matrix2D addColumn​(String colLabel,
                           Object[] colValues)
        Add a column to this Matrix2D, with the value on each row retreived from the given values array.
        Parameters:
        colLabel - Label of the new column
        colValues - Array of values to populate the new column with
        Returns:
        A reference to this matrix, to which the new column is added
      • addColumn

        Matrix2D addColumn​(String colLabel)
        Add a column to this Matrix2D, without any values
        Parameters:
        colLabel - Label of the new column
        Returns:
        A reference to this matrix, to which the new column is added
      • renameColumns

        Matrix2D renameColumns​(Map<String,​String> old2NewColLabels)
        Renames the columns in this matrix so that the old column label = key, and the new column label = value in the given map.
        Parameters:
        old2NewColLabels - Old --> new column label mapping
        Returns:
        A reference to this matrix, in which the matching columns have been renamed
      • replaceMissingValues

        Matrix2D replaceMissingValues​(String colLabel,
                                      Object val)
        Replaces nulls in the given column with the given value.
        Parameters:
        colLabel - Label of column to process
        val - Value to set
        Returns:
        A reference to this matrix, in which the null values have been replaced.
      • quantize

        List<String> quantize​(int numberOfBuckets,
                              String newColLabelPostfix,
                              String bucketNaming)
        Convenience method to quantize all numeric columns in one go.
        Parameters:
        numberOfBuckets -
        newColLabelPostfix - Example: the bins for column 'margin' go into a new column named 'margin_q' if the prefix is set to '_q'.
        bucketNaming -
      • quantize

        void quantize​(String colLabel,
                      int numberOfBuckets,
                      String newColLabel,
                      String bucketNaming)
        Performs equal width quantization of a scalar distribution in a given column, and maps each value to its nearest lower quantile. A column is added if none with the given label exists, or when the label is null.
        Parameters:
        colLabel - Label of the column holding the scalar values
        numberOfBuckets - Number of quantiles/buckets to calculate (max 100). Typically 10 deciles or 4 quartiles.
        colLabel - Label of the new column to add, holding the mapped quantile
        bucketNaming - One of the following values:

        • "ABC": quantiles are named "A","B","C" etc.
        • "INT": quantiles are numbered 0,2,3,...
        • "RANGE": quantile names reflect the range of the bucket (lower bound → upper bound)
        • "RANGE_PCT": as "RANGE" but not the bounds are formatted as percentages (ex. 8% vs 0.08)

      • toResultMatrix

        ResultMatrix toResultMatrix​(Map<String,​String> columnMapping)
        Create a ResultMatrix from a Matrix2D, selecting only the columns included in the mapping if such a mapping is provided.
        Parameters:
        columnMapping - Mapping ResultMatrix column name --> Matrix2D column label
        Returns:
        ResultMatrix
      • fromResultMatrix

        static Matrix2DImpl fromResultMatrix​(ResultMatrix resultMatrix)