Place potatoes into a large pot and cover with salted water; bring to a boil. Reduce heat to medium-low and simmer until tender, about 20 minutes. Drain, reserving 4 cups of the water from boiling.

Heat olive oil in a skillet over medium heat. Cook and stir onion, celery, and garlic in olive oil until tender, 3 to 5 minutes; add to potatoes with reserved water, milk, cream of celery soup, butter, and parsley. Season soup with salt and pepper.

Bring potato mixture to a simmer over low heat; stirring frequently, cook at a simmer for 30 minutes.

Cook and stir bacon in a skillet over medium-high heat until crisp, about 10 minutes. Drain grease.

Ladle soup into bowls. Sprinkle bacon and Cheddar cheese over soup to serve.