We were in Paris for a few days in the summer of 2019 with our two boys - a 6 year old and almost 2 year old(don't recommend) after 8 days in Provence. A baby might be easier than a toddler though.

I will say that while it is tricky to get through the Louvre with a stroller(lots of stairs and it's absolutely massive), you can get as close as you possibly can get to the Mona Lisa. Yes, it's a bit overrated, but being able to bypass the crowds and get all the way up to within a few feet of arguably the most famous painting in the world is pretty cool. There's a roped off walkway on the right side of the room that you can use to get to the front.

You get to go past everyone to the wooden rail and spend as much time there as you want - LINK

Also went to the D'orsay, which is a great museum. Much, much smaller than the Louvre. I'd recommend getting lunch there. It's an amazing room.

Another cool thing with kids is the Batobus. It's a water taxi that makes a big loop on the Seine - basically from the botanical gardens(Jardin de Plantes) to the Eiffel Tower and back with several stops along the way. I think it's about a 2 hour round trip and tickets are very reasonable. We were staying near the botanical gardens and ate dinner in the Eiffel Tower the last night(touristy I know, but the kids loved it) and rode back as the sun was setting. Really fun way to cap off the trip.

We did do a lot of walking along the Seine too. Mostly when our younger one would nap, so we'd just kind of stroll along and people watch. There are actually a few little playgrounds, which were very convenient. The 6 year old probably did 10+ miles in 2 days. The other days we used the Subway to get to Sacre Coeur and other areas not right on the Seine.

One of the most enjoyable things we did was getting a late lunch and strolling through/sitting in the Tulerie after the Louvre on a sunny day. We found a big tree to sit under and just enjoyed the afternoon for a while while we ate and our younger one napped.