Keywords:-
Article Content:-
Abstract
Cloud IaaS platforms readily provide access to homogeneous multi-core machines, whether they are physical ("bare metal") or virtual machines. Each of these machines can be equipped with high-performance SSD disks, enabling the distribution of workflow-generated files across multiple machines, which helps minimize the overhead associated with data transfers. In this paper, we propose a scheduling algorithm called SMDT-ERU (Scheduling for Minimizing Data Transfer - Enhancing Resource Utilization), designed to reduce the makespan of data-intensive workflows by minimizing data transfers between dependent tasks over the network. Intermediate files generated by tasks are stored locally on the disk of the machine where the tasks are executed.
Through experimentation, we confirm that increasing the number of cores per machine reduces the additional costs caused by network data transfers. Real-world workflow experiments demonstrate the advantages of the proposed algorithm. Our data-driven scheduling approach significantly reduces execution time and the volume of data transferred over the network, outperforming one of the leading state-of-the-art algorithms, which we have adapted to fit our assumptions.
References:-
References
X. Meng et L. Golab, « Parallel Scheduling of Data-Intensive Tasks », in Euro-Par 2020: Parallel Processing, M. Malawski et K. Rzadca, Éd., Cham: Springer International Publishing, 2020, p. 117‑133. doi: 10.1007/978-3-030-57675-2_8.
D. C. M. de Oliveira, J. Liu, et E. Pacitti, Data-Intensive Workflow Management. Springer Nature, 2022.
S. Pellegrini, F. Giacomini, et A. Ghiselli, « A Practical Approach for a Workflow Management System », in Grid Middleware and Services: Challenges and Solutions, D. Talia, R. Yahyapour, et W. Ziegler, Éd., Boston, MA: Springer US, 2008, p. 279‑287. doi: 10.1007/978-0-387-78446-5_18.
« Calcul — Types d’instances Amazon EC2 — AWS », Amazon Web Services, Inc. Consulté le: 16 octobre 2024. [En ligne]. Disponible sur: https://aws.amazon.com/fr/ec2/instance-types/
M. S. R. Krishna et S. Mangalampalli, « A Systematic Review on Various Task Scheduling Algorithms in Cloud Computing », EAI Endorsed Trans. Internet Things, vol. 10, 2024,
doi: 10.4108/eetiot.4548.
L. Yang, Y. Xia, X. Zhang, L. Ye, et Y. Zhan, « Classification-Based Diverse Workflows Scheduling in Clouds », IEEE Trans. Autom. Sci. Eng., vol. 21, no 1, p. 630‑641, janv. 2024,
doi: 10.1109/TASE.2022.3217666.
Z. Zhu et X. Tang, « Deadline-constrained workflow scheduling in IaaS clouds with multi-resource packing », Future Gener. Comput. Syst., vol. 101, p. 880‑893, déc. 2019,
doi: 10.1016/j.future.2019.07.043.
D. Alsadie et M. Alsulami, « Enhancing Workflow Efficiency: A Modified Firefly Algorithm for Hybrid Cloud-Edge Environments », 8 août 2024, Research Square. doi: 10.21203/rs.3.rs-4623299/v1.
S. Murad et al., « OPTIMIZED MIN-MIN TASK SCHEDULING ALGORITHM FOR SCIENTIFIC WORKFLOWS IN A CLOUD ENVIRONMENT », J. Theor. Appl. Inf. Technol., vol. 100, p. 480‑506, janv. 2022.
A. Patil et B. Thankachan, « Review on a comparative study of various task scheduling algorithm in cloud computing environment », Turk. J. Comput. Math. Educ. TURCOMAT, vol. 11, no 3, p. 1396‑1401, 2020.
O. Sukhoroslov, « Scheduling of Workflows with Task Resource Requirements in Cluster Environments », in Parallel Computing Technologies, V. Malyshkin, Éd., Cham: Springer Nature Switzerland, 2023, p. 177‑196.
doi: 10.1007/978-3-031-41673-6_14.
R. Akraminejad, N. Khaledian, A. Nazari, et M. Voelp, « A multi-objective crow search algorithm for optimizing makespan and costs in scientific cloud workflows (CSAMOMC) », Computing, vol. 106, no 6, p. 1777‑1793, juin 2024,
doi: 10.1007/s00607-024-01263-4.
S. Bansal et H. Aggarwal, « A multiobjective optimization of task workflow scheduling using hybridization of PSO and WOA algorithms in cloud-fog computing », Clust. Comput., vol. 27, no 8, p. 10921‑10952, nov. 2024, doi: 10.1007/s10586-024-04522-3.
M. Raeisi-Varzaneh, O. Dakkak, Y. Fazea, et M. G. Kaosar, « Advanced cost-aware Max–Min workflow tasks allocation and scheduling in cloud computing systems », Clust. Comput., vol. 27, no 9, p. 13407‑13419, déc. 2024, doi: 10.1007/s10586-024-04594-1.
Z. Sun, B. Zhang, C. Gu, R. Xie, B. Qian, et H. Huang, « ET2FA: A Hybrid Heuristic Algorithm for Deadline-Constrained Workflow Scheduling in Cloud », IEEE Trans. Serv. Comput., vol. 16, no 3, p. 1807‑1821, mai 2023,
doi: 10.1109/TSC.2022.3196620.
S. Mangalampalli, G. R. Karri, M. Kumar, O. I. Khalaf, C. A. T. Romero, et G. A. Sahib, « DRLBTSA: Deep reinforcement learning based task-scheduling algorithm in cloud computing », Multimed. Tools Appl., vol. 83, no 3, p. 8359‑8387, janv. 2024, doi: 10.1007/s11042-023-16008-2.
S. Kumar et S. Chander, « Comparative Analysis Of Task Scheduling Algorithms In Cloud Environment In Terms of Their Future Prospective And Risk », Webology, vol. 18, no 5, 2021, Consulté le: 15 octobre 2024. [En ligne]. Disponible sur: https://www.webology.org/data-cms/articles/20220212054937pmwebology%2018%20(5)%20-%2059%20pdf.pdf
P. Sadotra, P. Chouksey, M. Chopra, R. Koser, et R. Rawat, « Research Review on Task Scheduling Algorithm for Green Cloud Computing », in Scalable Modeling and Efficient Management of IoT Applications, IGI Global, 2025, p. 137‑152.
doi: 10.4018/979-8-3693-1686-3.ch007.
A. Taghinezhad-Niar, S. Pashazadeh, et J. Taheri, « QoS-aware online scheduling of multiple workflows under task execution time uncertainty in clouds », Clust. Comput., vol. 25, no 6, p. 3767‑3784, déc. 2022, doi: 10.1007/s10586-022-03600-8.
Z. Sun, C. Gu, H. Huang, et H. Zhang, « T2FA: A Heuristic Algorithm for Deadline-Constrained Workflow Scheduling in Cloud with Multicore Resource », in 2021 IEEE 14th International Conference on Cloud Computing (CLOUD), sept. 2021, p. 345‑354.
doi: 10.1109/CLOUD53861.2021.00048.
S. M. F. D. S. Mustapha et P. Gupta, « DBSCAN inspired task scheduling algorithm for cloud infrastructure », Internet Things Cyber-Phys. Syst., vol. 4, p. 32‑39, janv. 2024,
doi: 10.1016/j.iotcps.2023.07.001.
R. Sandhu, M. Faiz, H. Kaur, A. Srivastava, et V. Narayan, « Enhancement in performance of cloud computing task scheduling using optimization strategies », Clust. Comput., vol. 27, no 5, p. 6265‑6288, août 2024,
doi: 10.1007/s10586-023-04254-w.
H. Mikram, S. El Kafhali, et Y. Saadi, « HEPGA: A new effective hybrid algorithm for scientific workflow scheduling in cloud computing environment », Simul. Model. Pract. Theory, vol. 130, p. 102864, janv. 2024,
doi: 10.1016/j.simpat.2023.102864.
« https://pegasus.isi.edu/workflow_gallery/ », Pegasus WMS. Consulté le: 16 octobre 2024. [En ligne]. Disponible sur:
https://pegasus.isi.edu/workflow_gallery/
H. Casanova et al., « Developing accurate and scalable simulators of production workflow management systems with WRENCH », Future Gener. Comput. Syst., vol. 112, p. 162‑175, nov. 2020, doi: 10.1016/j.future.2020.05.030.
H. Casanova, R. Tanaka, W. Koch, et R. Ferreira Da Silva, « Teaching parallel and distributed computing concepts in simulation with WRENCH », J. Parallel Distrib. Comput., vol. 156, p. 53‑63, oct. 2021, doi: 10.1016/j.jpdc.2021.05.009.
H. Casanova, S. Pandey, J. Oeth, R. Tanaka, F. Suter, et R. Ferreira Da Silva, « WRENCH: A Framework for Simulating Workflow Management Systems », in 2018 IEEE/ACM Workflows in Support of Large-Scale Science (WORKS), Dallas, TX, USA: IEEE, nov. 2018, p. 74‑85.
doi: 10.1109/WORKS.2018.00013.
« SimGrid Home ». Consulté le: 16 octobre 2024. [En ligne]. Disponible sur: https://simgrid.org/
H. Topcuoglu, S. Hariri, et Min-You Wu, « Performance-effective and low-complexity task scheduling for heterogeneous computing », IEEE Trans. Parallel Distrib. Syst., vol. 13, no 3,
p. 260‑274, mars 2002, doi: 10.1109/71.993206.
« Advanced cost-aware Max–Min workflow tasks allocation and scheduling in cloud computing systems | Cluster Computing ». Consulté le: 18 octobre 2024. [En ligne]. Disponible sur: https://link.springer.com/article/10.1007/s10586-024-04594-1
G. Juve, A. Chervenak, E. Deelman, S. Bharathi, G. Mehta, et K. Vahi, « Characterizing and profiling scientific workflows », Future Gener. Comput. Syst., vol. 29, no 3, p. 682‑692, mars 2013,
doi: 10.1016/j.future.2012.08.015.