I think George Lucas would agree with you.

While he directed the first film, Empire and Jedi had other directors. When it came to the prequel series Lucas really tried to get someone else to direct, but everyone turned him down as the project was “too daunting”.

Lucas is best as the idea guy.