Source Code Analysis - How Custom Partition Commit Policies Are Created and Invoked in Flink’s FileSystemTableSinkWhen writing to a filesystem via Flink’s Table API or SQL with custom partitioning ena

Ingress and Route53 Issues with Druid AWS DeploymentsBackgroundWhen deploying Druid with Kubernetes on AWS, it is common to use AWS ALB (Application Load Balancer) for traffic routing and load balanci

Frame Size Configuration and RowTooLarge Error Resolution in DruidBackground In Apache Druid, data processing usually involves dividing large-scale data into multiple “frames” for processing. In this

HistoryServer IntroductionHistoryServer is an HTTP service built into Spark, we can start it with the following code. When HistoryServer is started, it starts two threads to parse and clean the log fi

Affinity-based Scheduling in KubernetesK8s POD is automatically scheduled to a NODE. By default, the scheduler considers whether resources are sufficient and tries to keep the load as evenly distribut

Probes Introduction Liveness probes: Liveness probes are used to determine whether a container is healthy. If the health condition is not met, kubelet will decide whether to restart the Pod based on t

BackgroundOur machines have been frequently triggering disk capacity alerts recently, and we found that the following directories were consuming a significant amount of disk space: 1sudo du -sh ./* --

BackgroundWhile deploying Druid on Kubernetes, I noticed that after changing the replica count of the StatefulSet, the pod remained stuck in Pending. Running kubectl describe on the pod surfaced the f

HDFS Decommission Codes and IntroductionBackground of HDFS decommissionCurrently there are 200+ nodes of 1.22PB HDFS3 and fewer than 100 nodes of 480T HDFS. The decommissioning of nodes is slow, migra