Pullup settings recommendations
To ensure optimal performance, efficient resource utilization, and successful job execution, you can fine-tune the job resource settings from the global job limits page. This is especially important when dealing with exceptionally large sets of data.
The following table shows recommendations on how to scale your job in different circumstances.
| Setting | 100,000 rows | 1,000,000 rows | 20,000,000 rows | 50,000,000 rows |
|---|---|---|---|---|
| Number of executors | ||||
| Executor cores | ||||
| Executor memory | ||||
| Driver memory | ||||
| Number of partitions | ||||
| Memory overhead |