Re: [coreboot] Question about PCIe separate reference clock(solved)

[Zheng Bao]

This problem is solved by setting the CPU internal clock as NON-SSC(spread spectrum clock).

PCIe spec says seperate ref clock should be Non-SSC. We only set device clock as Non-SSC before,

ignoring the clock at CPU side.

Now the system is stable. The"slot clock configuration" and "common clock configuration" are both still 1.

It doesn't seem to matter.

Thanks to you all.

Zheng

________________________________
From: Kyösti Mälkki 
Sent: Friday, January 13, 2017 4:58 PM
To: Zheng Bao
Cc: Predrag Vidic; [e-mail]; Zoran Stojsavljevic
Subject: Re: [coreboot] Question about PCIe separate reference clock

On Fri, Jan 13, 2017 at 6:01 PM, Zheng Bao <[e-mail]> wrote:

About "Asynchronous clock mode on mainboard side", I guess.

Both the device and bridge have the fields "slot clock configuration" and "common clock configuration".

Again assuming the mainboard side is CPU (some AMD SoC) and you are running coreboot and can bodify binaryPI / AGESA firmware.

You can spoof this in the firmware and leave "Common Clock Configuration" bit unset on both sides, even if they report "Slot Clock Configuration = 1". This affects the ASPM L0s and L1 latencies devices report in the Link Capabilities register, but may also change hardware behaviour since specs require link retraining after bit is set.

On our board, both the"slot clock configuration" of device (E8860)  and bridge are 1.

Does it mean the "on mainboard"  side it does not support "Asynchronous clock mode"?

So, any specs from the mainboard or CPU side of the link? Ultimately your 100MHz clock stability and jitter is affected by the choice of your 25MHz crystal part and PCB layout, right?

Kyösti

________________________________
From: Predrag Vidic <[e-mail]>
Sent: Friday, January 13, 2017 10:29 AM

To: Zheng Bao
Cc: Zoran Stojsavljevic; [e-mail]
Subject: Re: [coreboot] Question about PCIe separate reference clock

Hi Zheng,

Schematic is simple and OK. I assume you are trying to run Wolf VPXxx-E8860-xxxx or similar GPU borad. You can't do much there if there is a hardware problem but it is most possibly a software setup problem.

VPX does not have incomming clock so it is by default in async_clock mode. The question is did you allowed that mode on the main board side?

Asynchronous clock mode is not default mode for a motherboard. Check if the mode is allowed. Second, check if the other PCIe device can work on the motherboard PCIe slot.

Regards,
Predrag

On Fri, Jan 13, 2017 at 6:44 AM, Zheng Bao <[e-mail]> wrote:

Here is the ref clk part. Please review.

No refclk routes to VPX connector.

Thanks.

Zheng

________________________________
From: Predrag Vidic <[e-mail]>
Sent: Thursday, January 12, 2017 6:53 PM
To: Zheng Bao
Cc: Zoran Stojsavljevic; [e-mail]

Subject: Re: [coreboot] Question about PCIe separate reference clock

Hi Zheng,

Without knowing the particular solution for the PCIe transmitter you have on your board, I'd check refclk pins on your design for the proper termination. The problem can be in irregular readings from the pins.

Reragds,
Predrag

On Thu, Jan 12, 2017 at 3:26 PM, Zheng Bao <[e-mail]> wrote:

Q: do you use local xtal attached to   Si52111-B5 to generate local PCIe 25MHz clock?

Zheng: yes

Q: If you dothis, my next question is how you synchronize these two clocks: Local
PCIe 25 MHz and common reference clock from CPU?

Zheng: We do NOT sync these two.

Q: Since these two clocks, as I understand above scenario, are
asynchronous to each other?!

Zheng: yes. Asynchronous.

The VPX connector does not have a PCIe ref clock signal, so we can not send CPU PCIe ref clock to

device.  The PCIe spec says if the separate refclk on devices should be 100MHz ± 300PPM, with SSC

off.  We believe our board meet this requirement. So we doubt the problem lies in PCI configration space.

Zheng

________________________________
From: Zoran Stojsavljevic <[e-mail]>
Sent: Thursday, January 12, 2017 9:22 AM
To: Zheng Bao; Predrag Vidic
Cc: [e-mail]
Subject: Re: [coreboot] Question about PCIe separate reference clock

Hello Zheng,

For decades, I've been FW/SW engineer, but I do understand a little
bit of a HW. I have looked into the Si52111-B5 data sheet for
clarification.

My problem here is to understand, your use case: do you use local xtal
attached to   Si52111-B5 to generate local PCIe 25MHz clock? If you do
this, my next question is how you synchronize these two clocks: Local
PCIe 25 MHz and common reference clock from CPU?

Since these two clocks, as I understand above scenario, are
asynchronous to each other?!

Please, clarify for us your use case.

Thank you,
Zoran
_______

On 1/12/17, Zheng Bao <[e-mail]> wrote:
> Our VPX design uses separate reference clock source, which is Si52111-B5 (No
> spread), instead of common ref clock from CPU.
> Now The system is unstable. Reading PCIE configuration space is unstable
> too. (If we add some fly wire to make it work with common ref clock, the
> system becomes stable.)
>
> (abstracted from PCIe spec: 12 Slot Clock Configuration - This bit indicates
> that the
> component uses the same physical reference clock that the
> platform provides on the connector. If the device uses an
> independent clock irrespective of the presence of a reference
> clock on the connector, this bit must be clear.
> For a multi-Function device, each Function must report the
> same value for this bit.)
>
> Based on my understanding, the BIOS need to read bit "Slot Clock
> Configurationclear" to see if
> separate ref clock is used.  BIOS then write bit "Common Clock
> Configuration".
>
> On our board, the bit "Slot Clock Configuration" is always 1, which I assume
> should be 0.
>
> My question is, how the hardware affect the bit "Slot Clock Configuration"?
> How do we need to design our board to make the bit "Slot Clock
> Configuration" be 0?
>
> Thanks.
>
> Zheng