CPU Idle Cooling

CPU Idle Cooling

이 문서는 `GPL-2.0` SPDX license를 사용하며 thermal control을 위한 CPU idle injection cooling 방식을 설명합니다.

.. SPDX-License-Identifier: GPL-2.0

================
CPU Idle Cooling
================

Idle cooling이 필요한 상황

특정 상황에서 SoC는 critical temperature limit에 도달하고 temperature control point 주위로 온도를 안정화하지 못할 수 있습니다. Kernel은 cooling device를 작동시켜 dissipated power를 줄여야 하며, critical temperature에서 온도를 낮추는 결정은 performance에 영향을 줍니다.

Clock gating으로 component의 dynamic leakage를 최소로 낮춘 뒤에도 silicon temperature가 계속 오르는 thermal runaway가 발생할 수 있습니다. Static leakage 때문에 상승이 이어질 수 있으며, 유일한 해결책은 component를 power down하여 dynamic·static leakage를 모두 없애고 식히는 것입니다.

또 다른 문제는 system이 특정 power budget을 요구하지만 OPP density 때문에 요청보다 낮은 power의 OPP만 선택하여 CPU를 under-utilize하고 performance를 잃는 경우입니다.

즉 한 OPP는 요청 power budget보다 낮아 CPU를 충분히 쓰지 못하고, 다음 OPP는 budget을 초과합니다. 중간 OPP가 존재했다면 사용할 수 있었을 구간입니다.

Situation:
----------

Under certain circumstances a SoC can reach a critical temperature
limit and is unable to stabilize the temperature around a temperature
control. When the SoC has to stabilize the temperature, the kernel can
act on a cooling device to mitigate the dissipated power. When the
critical temperature is reached, a decision must be taken to reduce
the temperature, that, in turn impacts performance.

Another situation is when the silicon temperature continues to
increase even after the dynamic leakage is reduced to its minimum by
clock gating the component. This runaway phenomenon can continue due
to the static leakage. The only solution is to power down the
component, thus dropping the dynamic and static leakage that will
allow the component to cool down.

Last but not least, the system can ask for a specific power budget but
because of the OPP density, we can only choose an OPP with a power
budget lower than the requested one and under-utilize the CPU, thus
losing performance. In other words, one OPP under-utilizes the CPU
with a power less than the requested power budget and the next OPP
exceeds the power budget. An intermediate OPP could have been used if
it were present.

Idle cycle을 통한 해결

제어된 period 안에서 일정 duration 동안 static·dynamic leakage를 제거하면 SoC temperature를 낮출 수 있습니다. Idle state duration 또는 idle cycle injection period를 조절하여 power budget을 modulation하고 온도를 완화합니다.

OPP density는 CPUFreq control precision에 큰 영향을 줍니다. Vendor마다 OPP density가 다양하고 일부는 OPP 사이 power gap이 커서 thermal control에서는 performance를 잃고 다른 상황에서는 활용 가능한 power를 잃습니다.

특정 OPP에서 같은 cluster의 모든 CPU에 cluster idle state target residency보다 긴 idle cycle을 주입하면, 해당 시간 동안 state 진입 energy를 제외하고 static·dynamic leakage를 낮출 수 있다고 가정합니다.

Idle cycle을 포함한 sustainable power는 OPP sustainable power와 선형 관계이며 `Power(IdleCycle) = Coef x Power(OPP)` 같은 coefficient로 계산할 수 있습니다.

Solutions:
----------

If we can remove the static and the dynamic leakage for a specific
duration in a controlled period, the SoC temperature will
decrease. Acting on the idle state duration or the idle cycle
injection period, we can mitigate the temperature by modulating the
power budget.

The Operating Performance Point (OPP) density has a great influence on
the control precision of cpufreq, however different vendors have a
plethora of OPP density, and some have large power gap between OPPs,
that will result in loss of performance during thermal control and
loss of power in other scenarios.

At a specific OPP, we can assume that injecting idle cycle on all CPUs
belong to the same cluster, with a duration greater than the cluster
idle state target residency, we lead to dropping the static and the
dynamic leakage for this period (modulo the energy needed to enter
this state). So the sustainable power with idle cycles has a linear
relation with the OPP’s sustainable power and can be computed with a
coefficient similar to::

            Power(IdleCycle) = Coef x Power(OPP)

Idle injection과 duty cycle

Idle injection의 기본 개념은 각 control cycle마다 지정된 시간 동안 CPU를 강제로 idle state에 넣는 것입니다. CPUFreq 외에 CPU power와 heat를 제어하는 또 하나의 방법입니다.

같은 cluster의 모든 CPU가 idle cycle을 동기적으로 주입하면 cluster가 최소 power consumption의 power-down state에 도달해 static leakage를 거의 0으로 줄일 수 있습니다. 다만 deep sleep state에서 깨어나야 하므로 추가 latency가 생깁니다.

구현은 허용할 수 있는 performance penalty와 고정 latency를 제공하는 fixed idle duration을 사용합니다. Mitigation 강도는 idle injection duty cycle을 변화시켜 높이거나 낮춥니다.

Cooling device state 수는 duty-cycle percentage를 기반으로 합니다. Mitigation이 없으면 state는 0이고 duty cycle도 0%입니다.

Mitigation이 시작되면 governor policy가 시작 state를 선택합니다. Fixed idle duration과 cooling device state인 duty cycle을 알면 running duration을 계산할 수 있습니다. Governor가 state와 duty cycle을 바꾸면 cooling effect가 달라집니다.

원문의 timing diagram은 idle duration은 고정하면서 running 구간을 줄여 duty cycle을 25%에서 33%, 50%로 높이는 모습을 보여 줍니다.

Idle injection duration은 mitigation 시작 시 허용 가능한 latency보다 작거나 같아야 합니다. 이 값은 platform과 user experience의 reactivity·performance trade-off에 따라 달라지므로 명시해야 합니다.

동시에 thermal mitigation에 사용할 idle state의 target residency보다 길어야 합니다. 더 짧으면 state 진입 비용 때문에 오히려 energy를 더 소비합니다.

Idle Injection:
---------------

The base concept of the idle injection is to force the CPU to go to an
idle state for a specified time each control cycle, it provides
another way to control CPU power and heat in addition to
cpufreq. Ideally, if all CPUs belonging to the same cluster, inject
their idle cycles synchronously, the cluster can reach its power down
state with a minimum power consumption and reduce the static leakage
to almost zero.  However, these idle cycles injection will add extra
latencies as the CPUs will have to wakeup from a deep sleep state.

We use a fixed duration of idle injection that gives an acceptable
performance penalty and a fixed latency. Mitigation can be increased
or decreased by modulating the duty cycle of the idle injection.

::

     ^
     |
     |
     |-------                         -------
     |_______|_______________________|_______|___________

     <------>
       idle  <---------------------->
                    running

      <----------------------------->
              duty cycle 25%


The implementation of the cooling device bases the number of states on
the duty cycle percentage. When no mitigation is happening the cooling
device state is zero, meaning the duty cycle is 0%.

When the mitigation begins, depending on the governor's policy, a
starting state is selected. With a fixed idle duration and the duty
cycle (aka the cooling device state), the running duration can be
computed.

The governor will change the cooling device state thus the duty cycle
and this variation will modulate the cooling effect.

::

     ^
     |
     |
     |-------                 -------
     |_______|_______________|_______|___________

     <------>
       idle  <-------------->
                running

      <--------------------->
          duty cycle 33%


     ^
     |
     |
     |-------         -------
     |_______|_______|_______|___________

     <------>
       idle  <------>
              running

      <------------->
       duty cycle 50%

The idle injection duration value must comply with the constraints:

- It is less than or equal to the latency we tolerate when the
  mitigation begins. It is platform dependent and will depend on the
  user experience, reactivity vs performance trade off we want. This
  value should be specified.

- It is greater than the idle state’s target residency we want to go
  for thermal mitigation, otherwise we end up consuming more energy.

Power equation과 현실 보정

Thermal trip point에서 특정 temperature에 대응하는 sustainable power를 유지해야 하지만 현재 소비 power는 `Power = Capacitance x Voltage^2 x Frequency x Utilisation`이며 sustainable power보다 큽니다. 그렇지 않다면 system setup에 문제가 있습니다.

OPP를 바꾸지 않으므로 `Voltage`와 `Frequency`는 인위적으로 고정되고, `Capacitance`와 `Utilisation`도 고정값입니다. 두 항을 `Dynamic Power Coefficient (Cdyn)`로 묶으면 `Pdyn = Cdyn x Voltage^2 x Frequency`로 단순화됩니다.

Power allocator governor는 device tree에 정의된 sustainable power를 목표로 power를 낮추라고 요청합니다. Idle injection은 한 OPP에서 full power로 실행하는 시간과 idle 시간을 섞어 평균 target power `Ptarget`을 만듭니다.

평균식은 `P(opp)target = ((Trunning x P(opp)running) + (Tidle x P(opp)idle)) / (Trunning + Tidle)`입니다.

이를 정리하면 `Tidle = Trunning x ((P(opp)running / P(opp)target) - 1)`이며 CPU running period를 알 때 필요한 idle injection을 계산할 수 있습니다.

반대로 idle injection duration을 알고 있으면 `Trunning = Tidle / ((P(opp)running / P(opp)target) - 1)`로 running duration을 계산합니다.

Running power가 target power보다 작으면 time이 음수가 되므로 이 식은 power reduction에만 사용할 수 있습니다. 따라서 running power가 target보다 큰 높은 OPP가 필요합니다.

단순 demonstration은 static leakage, idle state wakeup latency인 entry+exit latency, idle state target residency를 무시합니다. Static leakage 값은 vendor에서 얻기 어려워 대부분 0이라고 가정하지만 식에 추가할 수 있습니다.

정확한 계산에는 wakeup latency를 포함해야 하고, injected idle duration이 target residency보다 길어야 energy 증가나 mitigation 반전을 피할 수 있습니다.

최종 보정식은 `Trunning = (Tidle - Twakeup) x (((P(opp)dyn + P(opp)static) - P(opp)target) / P(opp)target)`입니다.

Power considerations
--------------------

When we reach the thermal trip point, we have to sustain a specified
power for a specific temperature but at this time we consume::

 Power = Capacitance x Voltage^2 x Frequency x Utilisation

... which is more than the sustainable power (or there is something
wrong in the system setup). The ‘Capacitance’ and ‘Utilisation’ are a
fixed value, ‘Voltage’ and the ‘Frequency’ are fixed artificially
because we don’t want to change the OPP. We can group the
‘Capacitance’ and the ‘Utilisation’ into a single term which is the
‘Dynamic Power Coefficient (Cdyn)’ Simplifying the above, we have::

 Pdyn = Cdyn x Voltage^2 x Frequency

The power allocator governor will ask us somehow to reduce our power
in order to target the sustainable power defined in the device
tree. So with the idle injection mechanism, we want an average power
(Ptarget) resulting in an amount of time running at full power on a
specific OPP and idle another amount of time. That could be put in a
equation::

 P(opp)target = ((Trunning x (P(opp)running) + (Tidle x P(opp)idle)) /
                        (Trunning + Tidle)

  ...

 Tidle = Trunning x ((P(opp)running / P(opp)target) - 1)

At this point if we know the running period for the CPU, that gives us
the idle injection we need. Alternatively if we have the idle
injection duration, we can compute the running duration with::

 Trunning = Tidle / ((P(opp)running / P(opp)target) - 1)

Practically, if the running power is less than the targeted power, we
end up with a negative time value, so obviously the equation usage is
bound to a power reduction, hence a higher OPP is needed to have the
running power greater than the targeted power.

However, in this demonstration we ignore three aspects:

 * The static leakage is not defined here, we can introduce it in the
   equation but assuming it will be zero most of the time as it is
   difficult to get the values from the SoC vendors

 * The idle state wake up latency (or entry + exit latency) is not
   taken into account, it must be added in the equation in order to
   rigorously compute the idle injection

 * The injected idle duration must be greater than the idle state
   target residency, otherwise we end up consuming more energy and
   potentially invert the mitigation effect

So the final equation is::

 Trunning = (Tidle - Twakeup ) x
                (((P(opp)dyn + P(opp)static ) - P(opp)target) / P(opp)target )