目录
- 背景
- 详细设计及实现
- 趋势比对定义类 TrendCompare
- 趋势比对执行类
- 使用案例
背景
数据之间两两趋势比较在数据分析应用中是非常常见的应用场景,如下所示:
模拟考批次 | 班级 | 学生 | 语文 | 数学 | 英语 |
202302 | 三年一班 | 张小明 | 130 | 145 | 133 |
202302 | 三年一班 | 王二小 | 128 | 138 | 140 |
202302 | 三年一班 | 谢春花 | 136 | 142 | 139 |
202301 | 三年一班 | 张小明 | 132 | 140 | 128 |
202301 | 三年一班 | 王二小 | 125 | 146 | 142 |
202301 | 三年一班 | 谢春花 | 138 | 143 | 140 |
202212 | 三年一班 | 张小明 | 135 | 138 | 120 |
202212 | 三年一班 | 王二小 | 123 | 145 | 138 |
202212 | 三年一班 | 谢春花 | 136 | 140 | 142 |
现在有一个需求:各班级的每个学生在不同考试批次的各学科成绩的进退步情况,得出数据如下
模拟考批次 | 班级 | 学生 | 语文 | 数学 | 英语 |
202302与202301对比 | 三年一班 | 张小明 | -2 | 5 | 5 |
202302与202301对比 | 三年一班 | 王二小 | 3 | -8 | -2 |
202302与202301对比 | 三年一班 | 谢春花 | -2 | -1 | -1 |
202301与202212对比 | 三年一班 | 张小明 | -3 | 2 | 8 |
202301与202212对比 | 三年一班 | 王二小 | 2 | 1 | 4 |
202301与202212对比 | 三年一班 | 谢春花 | 2 | 3 | -2 |
详细设计及实现
趋势比对定义类 TrendCompare
public class TrendCompare {
/**
* 主体的字段列表(如三年一班的张小明,那么主体字段列表为 班级 + 学生姓名)
*/
private String[] subjectFields;
/**
* 在某个字段的(介词) 如三年一班张晓明在不同考试批次的成绩对比结果
*/
private String atField;
/**
* 参与趋势比较的字段集合
*/
private String[] compareFields;
/**
* 赋值映射集合:给结果数据中指定的key设置指定的值
*/
private Map<String, String> assignValMap;
public String[] subjectFields() {
return this.subjectFields;
}
public TrendCompare subjectFields(String... fields) {
this.subjectFields = fields;
return this;
}
public String atField() {
return this.atField;
}
public TrendCompare atField(String field) {
this.atField = field;
return this;
}
public String[] compareFields() {
return this.compareFields;
}
public TrendCompare compareFields(String... fields) {
this.compareFields = fields;
return this;
}
/**
* 赋值操作
*
* @param field
* @param valueEL 值表达式
* @return
*/
public TrendCompare assignVal(String field, String valueEL) {
if (assignValMap == null) {
assignValMap = new HashMap<>();
}
assignValMap.put(field, valueEL);
return this;
}
public Map<String, String> assignValMap() {
return this.assignValMap;
}
}
该类定义了如下属性:
- 主体的字段列表
- 介词字段
- 比对的字段列表
如:各班级的每个学生在不同考试批次的各学科成绩的进退步情况
上面的需求映射到定义类的结果如下:
- 主体的字段列表(班级、学生)
- 介词字段(考试批次)
- 比对的字段列表(各学科:语文、数学、英语)
趋势比对执行类
该类提供了一个供外部调用的方法如下
public static <T> List<T> compare(List<T> dataList, TrendCompare trendCompare) {
Map<String, List<T>> groupMap = group(dataList, null, trendCompare.subjectFields());
List<T> resultList = new ArrayList<>();
for (List<T> groupDataList : groupMap.values()) {
List<T> diffValueList = new ArrayList<>();
int size = groupDataList.size();
if (size > 1) {
for (int i = 0; i < size - 1; i++) {
//数据之间两两比较 diffValue = minuend - subtrahend
T minuend = groupDataList.get(i);
T subtrahend = groupDataList.get(i + 1);
T diffValue = minus(trendCompare.compareFields(), minuend, subtrahend);
//设置主体信息
if (trendCompare.subjectFields() != null) {
for (String subjectField : trendCompare.subjectFields()) {
setFieldValue(diffValue, subjectField, getFieldValue(minuend, subjectField));
}
}
//设置介词字段信息
String atField = trendCompare.atField();
if (StringUtils.isNotEmpty(atField)) {
setFieldValue(diffValue, atField, getFieldValue(minuend, atField) + "与" + getFieldValue(subtrahend, atField) + "对比增减");
}
diffValueList.add(diffValue);
}
}
if (diffValueList.size() > 0) {
T firstData = groupDataList.get(0);
Map<String, Object> valMap = new HashMap<>();
//指定的赋值集合进行赋值
if (trendCompare.assignValMap() != null) {
for (Map.Entry<String, String> stringStringEntry : trendCompare.assignValMap().entrySet()) {
String field = stringStringEntry.getKey();
if (!StringUtils.equalsAny(field, trendCompare.compareFields())) {
String valueEL = stringStringEntry.getValue();
valMap.put(field, executeSpEL(valueEL, firstData));
}
}
}
for (Map.Entry<String, Object> entry : valMap.entrySet()) {
for (T diffValue : diffValueList) {
setFieldValue(diffValue, entry.getKey(), entry.getValue());
}
}
}
resultList.addAll(diffValueList);
}
return resultList;
}
可以看到,该方法要求传入
- 数据集合
- 趋势比对定义
两个参数,并最终返回趋势比对后的结果集合。
该方法的内部逻辑可分为如下2个步骤:
- 按主体分组
- 分组后组内数据两两比对,并最终返回比对结果。
使用案例
假设有如下这样一组数据
定义一个学生类:
public class Student {
private String batch;
private String banji;
private String studentNo;
private String name;
private String sex;
private Double yuwen;
private Double math;
private Double english;
private Double physics;
//extra
private String maxScoreName1;
public Student(String batch, String banji, String studentNo, String name, String sex, Double yuwen, Double math, Double english, Double physics) {
this.batch = batch;
this.banji = banji;
this.studentNo = studentNo;
this.name = name;
this.sex = sex;
this.yuwen = yuwen;
this.math = math;
this.english = english;
this.physics = physics;
}
}
我们写一个方法,返回如上数据:
public List<Student> getDataList() {
List<Student> dataList = new ArrayList<>();
dataList.add(new Student("202302", "三年一班", "20001001", "张小明", "男", 130.0, 145.0, 133.0, 92.0));
dataList.add(new Student("202302", "三年一班", "20001002", "王二小", "男", 128.0, 138.0, 140.0, 98.0));
dataList.add(new Student("202302", "三年一班", "20001003", "谢春花", "女", 136.0, 142.0, 139.0, 95.0));
dataList.add(new Student("202302", "三年二班", "20002001", "冯世杰", "男", 129.0, 144.0, 138.0, 96.0));
dataList.add(new Student("202302", "三年二班", "20002002", "马功成", "男", 130.0, 132.0, 133.0, 98.0));
dataList.add(new Student("202302", "三年二班", "20002003", "魏翩翩", "女", 136.0, 142.0, 137.0, 92.0));
dataList.add(new Student("202301", "三年一班", "20001001", "张小明", "男", 132.0, 142.0, 134.0, 92.0));
dataList.add(new Student("202301", "三年一班", "20001002", "王二小", "男", 126.0, 136.0, 135.0, 94.0));
dataList.add(new Student("202301", "三年一班", "20001003", "谢春花", "女", 136.0, 145.0, 139.0, 95.0));
dataList.add(new Student("202301", "三年二班", "20002001", "冯世杰", "男", 124.0, 143.0, 148.0, 90.0));
dataList.add(new Student("202301", "三年二班", "20002002", "马功成", "男", 140.0, 133.0, 138.0, 90.0));
dataList.add(new Student("202301", "三年二班", "20002003", "魏翩翩", "女", 126.0, 136.0, 135.0, 92.0));
return dataList;
}
趋势比对定义并执行比对:
List<Student> dataList = getDataList();
TrendCompare trendCompare = new TrendCompare()
.subjectFields("banji", "name")
.atField("batch")
.compareFields("yuwen", "math", "english")
//.assignVal("batch", "'环比增减'")
;
List<Student> resultList = DataProcessUtil.compare(dataList, trendCompare);
for (Student result : resultList) {
System.out.println(JSON.toJSONString(result));
}
结果如下:
{"banji":"三年一班","batch":"202302与202301对比增减","english":-1.0,"math":3.0,"name":"张小明","yuwen":-2.0}
{"banji":"三年一班","batch":"202302与202301对比增减","english":5.0,"math":2.0,"name":"王二小","yuwen":2.0}
{"banji":"三年一班","batch":"202302与202301对比增减","english":0.0,"math":-3.0,"name":"谢春花","yuwen":0.0}
{"banji":"三年二班","batch":"202302与202301对比增减","english":-10.0,"math":1.0,"name":"冯世杰","yuwen":5.0}
{"banji":"三年二班","batch":"202302与202301对比增减","english":-5.0,"math":-1.0,"name":"马功成","yuwen":-10.0}
{"banji":"三年二班","batch":"202302与202301对比增减","english":2.0,"math":6.0,"name":"魏翩翩","yuwen":10.0}