Stamina is important in Genshin Impact. It dictates how far you can run, glide, or climb, but thankfully in combat it is only used to fuel your dodging, not your attacks. You will start off with a set stamina value, and it will not increase as you level up a character. Instead, you will need to track down the Statues of the Seven. These statues will allow you to rank up, increasing your stamina value each time you do.
To rank them up, you will need to find Anemoculus for the Statues in Mondstadt, and Geoculus for the Statues in Liyue. You can then offer these items to the statues to rank up and gain benefits, including an increased stamina value that will allow you to run further and climb higher.
You can find these items spread all over the map, and they will be marked by white star symbols on the minimap when you get close to them. Some of them will be awkward to get, and may involve some climbing, gliding, or simple puzzles. Just grab everyone that you see and make regular trips to the Statues to keep your stamina value going up.
While this will get you a permanent increase in your stamina, you can use different types of food to affect it as well. Eating a Barbatos Ratatouille or Cream Stew will decrease stamina drain for a while, while other types of food will increase the rate at which stamina is regained. Make sure you are experimenting with cooking in the game to take advantage of all the benefits of different types of food.