Introducing Pro-JobSync, Prost's distributed batch processing system.

Distributed Job
Scheduling
Solution

Comprehensive
System Support

High Availability
Solution

Various
Support Scopes

Job Progress
Monitoring

A distributed job scheduling solution based on modern architecture (MA) with core design goals of fast development, simple learning, lightweight, and easy scalability.

The JOB-SYNC distributed job scheduling system provides comprehensive system support from architecture design to data management, security, and monitoring.
This system enables efficient data processing and analysis through intuitive tools designed for problem-solving and user experience optimization.

JOB-SYNC is a high-availability solution for automating data processing and analysis tasks, capable of handling large-scale data through periodic or event-based distributed processing,
and it can be centrally managed.

Includes easy job management via web pages, dynamic management functions, high availability (HA), distributed processing and HA, registration centers, elastic scalability, various trigger strategies,
scheduling expiration strategies, blocking handling strategies, job timeout control, job failure retry and notifications, routing strategies, and job progress monitoring.


Main Features

Dynamic Management Features

Users can easily create, read, update, and delete jobs through the web page, change job statuses in real-time, and immediately stop running jobs.

High Availability (HA) of Scheduling Center and Distributed Execution of Executors

The scheduling center can be deployed as a cluster through a centralized design, achieving high availability. Additionally, Executor tasks are executed in a distributed manner and can be deployed as a cluster to ensure execution reliability.


Various Trigger and Scheduling Strategies

Executors automatically register tasks periodically, and the scheduling center detects and triggers execution. Manual entry of Executor addresses is also supported. When a new Executor is added or removed, jobs are redistributed in the next scheduling cycle. Additionally, Cron triggers, fixed interval triggers, API triggers, manual triggers, parent-child job triggers, scheduling expiration strategies, blocking handling strategies, and more are provided. This allows users to flexibly schedule jobs according to their requirements.


Failure Notifications and Retry Mechanism

When a job fails, notifications are provided via email, SMS, KakaoTalk, and other methods. Users can configure custom retry counts for failed jobs, improving system reliability and reducing error response times.

Job Progress Monitoring and Logs

Job progress can be monitored in real-time, and rolling real-time log functionality allows users to check scheduling results online and view execution logs in a rolling manner.


Architecture

현행 아키텍처
Current Architecture

Job Scheduler: A powerful web application for scheduling and managing jobs. Users can easily create, modify, and delete jobs, as well as monitor the state of batch executors. The logging function enables real-time tracking and analysis of system execution status, providing key insights. If a job execution fails, alerts can be configured to send immediate notifications. Additionally, job dependencies can be defined to automate complex workflows and scheduling.

Task Executor: Provides the essential runtime environment for job execution. Optimized for Java development environments, it enables developers to implement flexible batch processes. Each Task Handler executes specific job units and is distributed across multiple instances to ensure high availability. This system allows efficient job distribution and management when workload increases or the system scales. Task Executor helps developers manage complex data processing tasks effectively.

목표 아키텍처
Target Architecture

Vue.js: The Vue.js framework is applied to develop the client interface, implementing a responsive UI.

Prometheus & Grafana: Prometheus collects data for system monitoring, and Grafana visualizes it.

ELK Stack: Elasticsearch stores log data, Logstash collects logs, and Kibana visualizes the data for log management and troubleshooting.

Kubernetes: Automates the deployment, scaling, and management of containerized applications.

Docker: Enhances scalability and system stability through outscaling and load balancing.


User Interface

??contents.service.jobsync.dashboard_reports_en??

Job Status: Provides a real-time overview of the system by categorizing all registered jobs into running, scheduled, completed, and disabled.

Processing History: Shows the trend of job processing over a specific period. Different colors visualize the job statuses such as success, failure, and running over time.

System Health: Allows real-time monitoring of key components such as application, database, disk, and external servers along with server uptime.

System Status: Provides real-time usage rates of key system resources such as CPU, memory, and disk, helping to detect potential bottlenecks or overloads in advance.

General Jobs

Job List: Displays all registered jobs in a list view, allowing you to quickly see the job name, schedule type, handler, and status.

Schedule Type: Identifies whether each job is manually triggered (NONE) or scheduled automatically (CRON), showing its execution cycle.

Handler / GLUE Type: Shows the handler class name and execution method (e.g., BEAN) connected to each job for better understanding of execution logic.

Status Control Switch: Allows toggling the standby state (on/off) for each job to quickly control its activation.

Search & Filter: Easily locate and manage jobs using group selection, keyword search, and status-based filters.

Handler Management

Handler List: Displays all registered handlers in a list format along with their associated job names and descriptions.

Title / Batch Name: Describes the purpose or testing intent of the handler, and shows the batch name to identify the execution environment.

Search & Filter: Quickly find the desired handler using name search and job group filters.

Last Modified: Shows when the handler was last updated, helping to track recent changes.

Job Execution Status

Job List Panel: View execution status, handler name, and author for each job, and select a job to see detailed execution information.

Execution Summary: Summarizes average CPU and memory usage, recent success rate, average execution time, and last execution time of the selected job.

Execution History Tab: Check start/end times, duration (in seconds), and result (success/failure) of each execution to identify issues quickly.

Performance/Error/Log Tabs: View detailed performance metrics, error history, server logs, and handler logs related to job execution through separate tabs.

User Activity Audit

Audit List: Displays user ID, name, action type (e.g., view, page visit), action details, API path, execution time, and IP address in a tabular format.

Search Filter: Quickly retrieve logs using date range, action type, keyword search, and group filters.

View Parameters: Allows detailed analysis of user requests by showing parameters passed with each request.

Audit Trail: Logs key system access events including login, enabling security auditing, anomaly detection, and user accountability tracking.

Server Resource Report

Handler List: Displays all registered handlers in a list format along with their associated job names and descriptions.

Title / Batch Name: Describes the purpose or testing intent of the handler, and shows the batch name to identify the execution environment.

Search & Filter: Quickly find the desired handler using name search and job group filters.

Last Modified: Shows when the handler was last updated, helping to track recent changes.

개인정보처리방침

  1. 수집하는 개인정보의 항목 및 수집방법
    1. 수집항목
    2. 수집방법
    3. DSA Full Course
  2. 개인정보의 수집 및 이용목적
  3. 수집한 개인정보의 제공
  4. 수집한 개인정보의 보유
    1. 프로스트 정책에 따른 정보 보유
    2. 관련 법령에 따른 정보의 보유
  5. 수집한 개인정보의 국외이전
  6. 개인정보의 파기 절차 및 방법
  7. 개인정보 자동 수집 장치의 설치ㆍ운영 및 그 거부에 관한 사항
  8. 개인정보의 안전성 확보 조치
  9. 이용자의 권리와 그 행사방법
  10. 개인정보보호책임자 및 고객서비스에 관한 사항
    1. 고객서비스 담당부서
    2. 개인정보보호책임자
  11. 고지의 의무

이메일주소무단수집거부

본 홈페이지에 게시된 이메일 주소가 전자우편 수집 프로그램이나 그 밖의 기술적 장치를 이용하여 무단으로 수집되는 것을 거부하며, 이를 위반 시 정보통신망법에 의해 형사 처벌됨을 유념하시기 바랍니다.

게시일: 2023년 3월 25일

Credits

Home
  1. DevOpsSec: Image by Wayhomestudio on Freepik
  2. Mis: Image by Wayhomestudio on Freepik
  3. PC/Mobile: Image by Benzoix on Freepik
  4. Specialist: Image by Freepik
  5. Team: Image by rawpixel.com on Freepik
  6. Software: Image by rawpixel.com on Freepik
Membership
  1. Login: Image by rawpixel.com on Freepik
  2. Join: Image by rawpixel.com on Freepik
Company Intro
  1. About: Image by Pixabay
  2. Profile: Image by Image by rawpixel.com on Freepik
Culture
  1. Work Life: Image by Andrey Popov on Adobe Stock
  2. Heal Family: Image by Rawpixel.com on Adobe Stock
  3. Horizontal: Image by Pixabay
  4. Trust: Image by Jon Anders Wiken on Adobe Stock
MIS/ERP/EIP
  1. ERP: Image by Freepik
  2. EIP: Image by Freepik
DevSecOps
  1. Security: Image by rawpixel.com on Freepik
  2. Automation: Image by Freepik
  3. Monitoring: Image by rawpixel.com on Freepik
  4. Culture: Image by Murrstock on Adobe Stock
Web/Mobile
  1. Front-end: Image by Freepik
  2. Back-end: Image by Freepik
  3. Database: Image by DCStudio on Freepik
  4. API: Image by Freepik
Recruit Notice
  1. Join Our Team: Image by Freepik