tlseparation.utility package

Submodules

tlseparation.utility.cloud_analysis module

tlseparation.utility.cloud_analysis.detect_nn_dist(arr, knn, sigma=1)

Calcuates the optimum distance among neighboring points.

Parameters:

arr : array

N-dimensional array (m x n) containing a set of parameters (n) over a set of observations (m).

knn : int

Number of nearest neighbors to search to constitue the local subset of points around each point in ‘arr’.

Returns:

dist : float

Optimal distance among neighboring points.

tlseparation.utility.cloud_analysis.detect_optimal_knn(arr, rad_lst=[0.1, 0.2, 0.3], sample_size=10000)

Detects optimal values for knn in order to facilitate material separation.

Parameters:

arr: array

Set of 3D points.

rad_lst: list

Set of radius values to generate samples of neighborhoods. This is used to select points to calculate a number of neighboring points distribution from the point cloud.

sample_size: int

Number of points in arr to process in order to genrate a distribution.

Returns:

knn_lst: list

Set of k-nearest neighbors values.

tlseparation.utility.cloud_analysis.detect_rad_nn(arr, rad)

Calculates an average of number of neighbors based on a fixed radius around each point in a point cloud.

Parameters:

arr : array

Three-dimensional (m x n) array of a point cloud, where the coordinates are represented in the columns (n) and the points are represented in the rows (m).

rad : float

Radius distance to select neighboring points.

Returns:

mean_knn : int

Average number of points inside a radius ‘rad’ around each point in ‘arr’.

tlseparation.utility.clustering module

tlseparation.utility.clustering.connected_component(arr, voxel_size)

Performs a connected component analysis to cluster points from a point cloud.

Parameters:

arr : array

Three-dimensional (m x n) array of a point cloud, where the coordinates are represented in the columns (n) and the points are represented in the rows (m).

voxel_size: float

Distance used to generate voxels from point cloud in order to perform the connected component analysis in 3D space.

Returns:

point_labels : array

1D array with cluster labels assigned to each point from the input point cloud.

tlseparation.utility.data_utils module

tlseparation.utility.data_utils.apply_nn_value(base, arr, attr)

Upscales a set of attributes from a base array to another denser array.

Parameters:

base : array

Base array to which the attributes to upscale were originaly matched.

arr : array

Target array to which the attributes will be upscaled.

attr : array

Attributes to upscale.

Returns:

new_attr : array

Upscales attributes.

Raises:

AssertionError:

length (number of samples) of “base” and “attr” must be equal.

tlseparation.utility.data_utils.entries_to_remove(entries, d)

Function to remove selected entries (key and respective values) from a given dict. Based on a reply from the user mattbornski [1] at stackoverflow.

Parameters:

entries : array

Set of entried to be removed.

d : dict

Dictionary to apply the entried removal.

References

[1]	mattbornski, 2012. http://stackoverflow.com/questions/8995611/removing-multiple-keys-from-a-dictionary-safely

tlseparation.utility.data_utils.get_diff(arr1, arr2)

Performs the intersection of two arrays, returning the entries not intersected between arr1 and arr2.

Parameters:

arr1 : array

N-dimensional array of points to intersect.

arr2 : array

N-dimensional array of points to intersect.

Returns:

arr : array

Difference array between ‘arr1’ and ‘arr2’.

tlseparation.utility.data_utils.remove_duplicates(arr, return_ids=False)

Removes duplicated rows from an array.

Parameters:

arr : array

N-dimensional array (m x n) containing a set of parameters (n) over a set of observations (m).

return_ids: bool

Option to return indices of duplicated entries instead of new array with unique entries.

Returns:

unique : array

N-dimensional array (m* x n) containing a set of unique parameters (n) over a set of unique observations (m*).

tlseparation.utility.downsampling module

tlseparation.utility.downsampling.downsample_cloud(point_cloud, downsample_size, return_indices=False, return_neighbors=False)

Downsamples a point cloud by voxelizing it and selecting points closest to the median coordinate of all points inside each voxel. The remaining points can be stored and returned as a dictrionary for later use in upsampling back to original input data.

Parameters:

point_cloud : numpy.ndarray

Three-dimensional (m x n) array of a point cloud, where the coordinates are represented in the columns (n) and the points are represented in the rows (m).

downsample_size : float

Size of the voxels used to sample points into groups and select the most central point from. Note that this will not be the final points distance from each other, but an approximation.

return_indices : bool

Option to return results as downsampled array (False) or the indices of downsampled points from original point cloud (True).

return_neighbors : bool

Option to return original neighbors of downsampled points (True) or not (False). This information can be used to upsample back the downsampled indices.

tlseparation.utility.downsampling.upsample_cloud(upsample_ids, neighbors_dict)

Upsample cloud based on downsampling information from ‘downsample_cloud’. This function will loop over each ‘upsample_ids’ and retrieve its original neighboring points stored in ‘neighbors_dict’.

Parameters:

upsample_ids : list

List of indices in ‘neighbors_dict’ to upsample.

neighbors_dict : dict

Neighbors information provided by ‘downsample_cloud’ containing all the original neighboring points to each point in the downsampled cloud.

Returns:

upsampled_indices : numpy.ndarray

Upsampled points from original point cloud.

tlseparation.utility.filtering module

tlseparation.utility.filtering.array_majority(arr_1, arr_2, **kwargs)

Applies majority filter on two arrays.

Parameters:

arr_1 : array

n-dimensional array of points to filter.

arr_2 : array

n-dimensional array of points to filter.

**knn : int or float

Number neighbors to select around each point in arr in order to apply the majority criteria.

**rad : int or float

Search radius arount each point in arr to select neighbors in order to apply the majority criteria.

Returns:

c_maj_1 : array

Boolean mask of filtered entries of same class as input ‘arr_1’.

c_maj_2 : array

Boolean mask of filtered entries of same class as input ‘arr_2’.

Raises:

AssertionError:

Raised if neither ‘knn’ or ‘rad’ arguments are passed with valid values (int or float).

tlseparation.utility.filtering.class_filter(arr_1, arr_2, target, **kwargs)

Function to apply class filter on an array based on the combination of classed from both arrays (arr_1 and arr_2). Which array gets filtered is defined by ‘’target’‘.

Parameters:

arr_1 : array

n-dimensional array of points to filter.

arr_2 : array

n-dimensional array of points to filter.

target : int or float

Number of the input array to filter. Valid values are 0 or 1.

**knn : int or float

Number neighbors to select around each point in arr in order to apply the majority criteria.

**rad : int or float

Search radius arount each point in arr to select neighbors in order to apply the majority criteria.

Returns:

c_maj_1 : array

Boolean mask of filtered entries of same class as input ‘arr_1’.

c_maj_2 : array

Boolean mask of filtered entries of same class as input ‘arr_2’.

Raises:

AssertionError:

Raised if neither ‘knn’ or ‘rad’ arguments are passed with valid values (int or float).

AssertionError:

Raised if ‘target’ variable is not an int or float with value 0 or 1.

tlseparation.utility.filtering.cluster_features(arr, labels, feature_threshold, min_pts=10)

Filters a set of connected components by a geometric feature threshold. This feature (n 2 in the separation methodology) is used to describe elongated shapes. If the shape of the cluster is elongated enough (i.e. feature value larger than threshold) the points belonging to this cluster are masked as True.

Parameters:

arr : array

Three-dimensional (m x n) array of a point cloud, where the coordinates are represented in the columns (n) and the points are represented in the rows (m).

labels : array

1D array with cluster labels assigned to each point from the input point cloud.

feature_threshold : float

Minimum feature value for the cluster to be set as elongated (True).

min_pts : int

Minimum number of points for the cluster to be set as valid (True).

Returns:

filter_mask : array

1D mask array setting True for valid poins in ‘arr’ and False otherwise.

tlseparation.utility.filtering.cluster_filter(arr, max_dist, eval_threshold)

Applies a cluster filter to a point cloud ‘arr’. This filter aims to remove small, isolated, clusters of points.

Parameters:

arr : array

Point cloud of shape n points x m dimensions to be filtered.

max_dist : float

Maximum distance between points to considered part of the same cluster.

eval_threshold : float

Minimum value for largest eigenvalue for a valid cluster. This value is an indication of cluster shape, in which the higher the eigenvalue, more elongated is the cluster. Points from clusters that have eigenvalue smaller then eval_threshold are filtered out.

Returns:

mask : array

Boolean mask of filtered points. Entries are set as True if belonging to a valid cluster and False otherwise.

tlseparation.utility.filtering.cluster_size(arr, labels, min_size)

Filters a set of connected components by maximum size on any dimension.

Parameters:

arr : array

Three-dimensional (m x n) array of a point cloud, where the coordinates are represented in the columns (n) and the points are represented in the rows (m).

labels : array

1D array with cluster labels assigned to each point from the input point cloud.

min_size : int/float

Minimum size, on any dimension, for a cluster to be set as valid (True)

Returns:

filter_mask : array

1D mask array setting True for valid poins in ‘arr’ and False otherwise.

tlseparation.utility.filtering.continuity_filter(wood, leaf, rad=0.05)

Function to apply a continuity filter to a point cloud that contains gaps defined as points from a second point cloud. This function works assuming that the continuous variable is the wood portion of a tree point cloud and the gaps in it are empty space or missclassified leaf data. In this sense, this function tries to correct gaps where leaf points are present.

Parameters:

wood : array

Wood point cloud to be filtered.

leaf : array

Leaf point cloud, with points that may be causing discontinuities in the wood point cloud.

rad : float

Radius to search for neighboring points in the iterative process.

Returns:

wood : array

Filtered wood point cloud.

not_wood : array

Remaining point clouds after the filtering.

tlseparation.utility.filtering.dist_majority(arr_1, arr_2, **kwargs)

Applies majority filter on two arrays.

Parameters:

arr_1 : array

n-dimensional array of points to filter.

arr_2 : array

n-dimensional array of points to filter.

**knn : int or float

Number neighbors to select around each point in arr in order to apply the majority criteria.

**rad : int or float

Search radius arount each point in arr to select neighbors in order to apply the majority criteria.

Returns:

c_maj_1 : array

Boolean mask of filtered entries of same class as input ‘arr_1’.

c_maj_2 : array

Boolean mask of filtered entries of same class as input ‘arr_2’.

Raises:

AssertionError:

Raised if neither ‘knn’ or ‘rad’ arguments are passed with valid values (int or float).

tlseparation.utility.filtering.feature_filter(arr, feature_id, threshold, knn)

Filters a point cloud based on a given feature threshold. Only points with selected feature values higher than threshold are kept as valid.

Parameters:

arr : array

Three-dimensional (m x n) array of a point cloud, where the coordinates are represented in the columns (n) and the points are represented in the rows (m).

feature_id : int

Column index of feature selected as criteria to filter. Column indices follow Python notation [0 - (n_columns - 1)].

threshold : float

Minimum feature value for valid points.

knn : int

Number of neighbors to select around each point. Used to describe local point arrangement.

Returns:

mask_feature : numpy.ndarray

Boolean mask with valid points entries set as True.

tlseparation.utility.filtering.plane_filter(arr, rad, threshold)

Filters a point cloud based on its points planicity. Removes points that are part of a neighbohood with planar spatial arrangement (low curvature).

Parameters:

arr : array

Three-dimensional (m x n) array of a point cloud, where the coordinates are represented in the columns (n) and the points are represented in the rows (m).

rad : float

Search radius distance around each point. Used to describe local point arrangement.

threshold : float

Minimum curvature value for valid points.

Returns:

mask_plane : numpy.ndarray

Boolean mask with valid points entries set as True.

tlseparation.utility.filtering.radius_filter(arr, radius, min_points)

Applies a radius search filter, which remove isolated points/clusters of points.

Parameters:

arr : array

Point cloud of shape n points x m dimensions to be filtered.

radius : float

Search radius around each point to form a neighborhood.

min_point : int

Minimum number of points in a neighborhood for it to be considered valid, i.e not filtered out.

Returns:

mask : array

Array of bools masking valid points as True and “noise” points as False.

tlseparation.utility.knnsearch module

tlseparation.utility.knnsearch.set_nbrs_knn(arr, pts, knn, return_dist=True, block_size=100000)

Function to create a set of nearest neighbors indices and their respective distances for a set of points. This function uses a knn search and sets a limit size for a block of points to query. This makes it less efficient in terms of processing time, but avoids running out of memory in cases of very dense/large arrays/queries.

Parameters:

arr : array

N-dimensional array to perform the knn search on.

pts : array

N-dimensional array to search for on the knn search.

knn : int

Number of nearest neighbors to search for.

return_dist : boolean

Option to return or not the distances of each neighbor.

block_size : int

Limit of points to query. The variable ‘pts’ will be subdivided in n blocks of size block_size to perform query.

Returns:

indices : array

Set of neighbors indices from ‘arr’ for each entry in ‘pts’.

distance : array

Distances from each neighbor to each central point in ‘pts’.

tlseparation.utility.knnsearch.set_nbrs_rad(arr, pts, rad, return_dist=True, block_size=100000)

Function to create a set of nearest neighbors indices and their respective distances for a set of points. This function uses a radius search and sets a limit size for a block of points to query. This makes it less efficient in terms of processing time, but avoids running out of memory in cases of very dense/large arrays/queries.

Parameters:

arr : array

N-dimensional array to perform the radius search on.

pts : array

N-dimensional array to search for on the knn search.

rad : float

Radius of the NearestNeighbors search.

return_dist : boolean

Option to return or not the distances of each neighbor.

block_size : int

Limit of points to query. The variable ‘pts’ will be subdivided in n blocks of size block_size to perform query.

Returns:

indices : array

Set of neighbors indices from ‘arr’ for each entry in ‘pts’.

distance : array

Distances from each neighbor to each central point in ‘pts’.

tlseparation.utility.knnsearch.subset_nbrs(distance, indices, new_knn, block_size=100000)

Performs a subseting of points from the results of a nearest neighbors search. This function assumes that the first index/distance in each row represents the center point of the neighborhood represented by said rows.

Parameters:

distance : array

Distances from each neighbor to each central point in ‘pts’.

indices : array

Set of neighbors indices from ‘arr’ for each entry in ‘pts’.

new_knn : array

Number of neighbors to select from the initial number of neighbors.

block_size : int

Limit of points to query. The variables ‘distance’ and ‘indices’ will be subdivided in n blocks of size block_size to perform query.

Returns:

distance : array

Subset of distances from each neighbor ‘indices’.

indices : array

Subset of neighbors indices from ‘indices’.

tlseparation.utility.peakdetect module

% Eli Billauer, 3.4.05 (Explicitly not copyrighted). % This function is released to the public domain; Any use is allowed.

Modifications in docstrings were performed by TLSepartion project to improve autodocumentation using Sphinx. All credits are still to Eli Billauer.

tlseparation.utility.peakdetect.peakdet(v, delta, x=None)

Converted from MATLAB script at http://billauer.co.il/peakdet.html

Parameters:

v: array

Input vector (1D array) of values.

delta: float

Value change that characterizes a peak. A point is considered a maximum peak if it has the maximal value, and was preceded (to the left) by a value lower by delta.

x: array

Set of x values to replace indices in maxtab/mintab.

Returns:

maxtab: array

2D array containing maxima peaks indices and values.

mintab: array

2D array containing minima peaks indices and values.

Notes

Eli Billauer, 3.4.05 (Explicitly not copyrighted). This function is released to the public domain; Any use is allowed.

tlseparation.utility.shortpath module

tlseparation.utility.shortpath.add_nodes(G, base_node, indices, distance, threshold)

Adds a set of nodes and weighted edges based on pairs of indices between base_node and all entries in indices. Each node pair shares an edge with weight equal to the distance between both nodes.

Parameters:

G : networkx graph

NetworkX graph object to which all nodes/edges will be added.

base_node : int

Base node’s id to be added. All other nodes will be paired with base_node to form different edges.

indices : list or array

Set of nodes indices to be paired with base_node.

distance : list or array

Set of distances between all nodes in ‘indices’ and base_node.

threshold : float

Edge distance threshold. All edges with distance larger than ‘threshold’ will not be added to G.

tlseparation.utility.shortpath.array_to_graph(arr, base_id, kpairs, knn, nbrs_threshold, nbrs_threshold_step, graph_threshold=inf)

Converts a numpy.array of points coordinates into a Weighted BiDirectional NetworkX Graph. This funcions uses a NearestNeighbor search to determine points adajency. The NNsearch results are used to select pairs of points (or nodes) that have a common edge.

Parameters:

arr : array

n-dimensional array of points.

base_id : int

Index of base id (root) in the graph.

kpairs : int

Number of points around each point in arr to select in order to build edges.

knn : int

Number of neighbors to search around each point in the neighborhood phase. The higher the better (careful, it’s memory intensive).

nbrs_threshold : float

Maximum valid distance between neighbors points.

nbrs_threshold_step : float

Distance increment used in the final phase of edges generation. It’s used to make sure that in the end, every point in arr will be translated to nodes in the graph.

graph_threshold : float

Maximum distance between pairs of nodes (edge distance) accepted in the graph generation.

Returns:

G : networkx graph

Graph containing all points in ‘arr’ as nodes.

tlseparation.utility.shortpath.extract_path_info(G, base_id, return_path=True)

Extracts shortest path information from a NetworkX graph.

Parameters:

G : networkx graph

NetworkX graph object from which to extract the information.

base_id : int

Base (root) node id to calculate the shortest path for all other nodes.

return_path : boolean

Option to select if function should output path list for every node in G to base_id.

Returns:

nodes_ids : list

Indices of all nodes in graph G.

distance : list

Shortest path distance (accumulated) from all nodes in G to base_id node.

path_list : dict

Dictionary of nodes that comprises the path of every node in G to base_id node.

tlseparation.utility.voxels module

tlseparation.utility.voxels.voxelize_cloud(arr, voxel_size)

Generates a dictionary of voxels containing their central coordinates and indices of points belonging to each voxel.

Parameters:

arr: array

Array of points/entries to voxelize.

voxel_size: float

Length of all voxels sides/edges.

Returns:

vox: defaultdict

Dictionary containing voxels. Keys are voxels’ central coordinates and values are indices of points in arr inside each voxel.

Module contents