Attachment #394133 for bug #201608

View Details Raw Unified Return to Bug 201608

| Differences between



2020-03-20  Daniel Bates  <dabates@apple.com>

        Replace "deferred element focus" functionality with alternative solution
        https://bugs.webkit.org/show_bug.cgi?id=201608

        Reviewed by Wenson Hsieh.

        This is a partial revert of r190278. Have the web process perform a layout, if needed, when
        computing focus element information and send an editor state update immediately. If layout is
        not needed then explicitly schedule a full editor state update.

        Currently, fetching focus element information neither sends an editor state update nor
        schedules one. As a result, when the web process tells the UI process to focus an element the
        UI process may need to defer doing so if the last received update did not include details
        that require up-to-date layout (e.g. the bounding rect of the focused element, which is used
        to scroll and zoom to center the focused element). The UI process then schedules an async message
        to the web process to fetch the full editor state, which will arrive in a layer tree commit message
        from the web process. (Note that the UI process schedules this request to ensure the web process
        knows that it is waiting for a layer tree commit. The web process can use this info to expedite
        a layer tree commit, if needed). This deferral mechanism complicates the element focusing and
        defocusing logic in the UI process and prevents fixing <https://bugs.webkit.org/show_bug.cgi?id=199960>.
        Instead remove this deferral concept and have the web process ensure that a full editor state
        update is sent or will be sent when computing the focus element information.

        * UIProcess/WebPageProxy.cpp:
        (WebKit::WebPageProxy::resetStateAfterProcessExited):
        * UIProcess/WebPageProxy.h:
        * UIProcess/ios/WebPageProxyIOS.mm:
        (WebKit::WebPageProxy::didCommitLayerTree):
        (WebKit::WebPageProxy::elementDidFocus):
        (WebKit::WebPageProxy::elementDidBlur):
        Remove bookkeeping code to track a deferred focus event or to perform the deferred event
        on layer tree commit.

        * WebProcess/WebPage/WebPage.cpp:
        (WebKit::WebPage::elementDidFocus):
        * WebProcess/WebPage/ios/WebPageIOS.mm:
        (WebKit::WebPage::getFocusedElementInformation): Save off whether a layout is needed then
        tell the page to layout if needed. If a layout was needed then send an editor state update
        immediately (it's an async message): this update will be a "full editor state" update that
        includes up-to-date layout details. Otherwise, schedule a full editor state update. While I
        am here, I updated the code to take out a ref on the focused frame's document before performing
        a layout because layout can cause arbitrary JavaScript execution that could detach the document
        from its frame view as part of destroying the document. Document destruction is detected by
        checking whether the document has been detached from its frame view. If this happens then
        bail out as there is no need to get focus element info.

2020-03-19  Daniel Bates  <dabates@apple.com>

        Have insertDictatedTextAsync() take an InsertTextOptions




#if PLATFORM(IOS_FAMILY)
    m_waitingForPostLayoutEditorStateUpdateAfterFocusingElement = false;
    m_deferredElementDidFocusArguments = nullptr;

    m_isVisibleActivity = nullptr;
    m_isAudibleActivity = nullptr;
    m_isCapturingActivity = nullptr;



typedef GenericCallback<const WebCore::IntPoint&, uint32_t, uint32_t> TouchesCallback;
typedef GenericCallback<const Vector<WebCore::SelectionRect>&> SelectionRectsCallback;
typedef GenericCallback<const FocusedElementInformation&> FocusedElementInformationCallback;
struct ElementDidFocusArguments {
    WTF_MAKE_STRUCT_FAST_ALLOCATED;
    FocusedElementInformation information;
    bool userIsInteracting;
    bool blurPreviousNode;
    OptionSet<WebCore::ActivityState::Flag> activityStateChanges;
    RefPtr<API::Object> userData;
};
#endif

#if PLATFORM(COCOA)


#if PLATFORM(IOS_FAMILY)
    Function<bool()> m_deviceOrientationUserPermissionHandlerForTesting;
    std::unique_ptr<ElementDidFocusArguments> m_deferredElementDidFocusArguments;
    bool m_waitingForPostLayoutEditorStateUpdateAfterFocusingElement { false };
    WebCore::FloatSize m_maximumUnobscuredSize;
    bool m_lastObservedStateWasBackground { false };



        m_hitRenderTreeSizeThreshold = true;
        didReachLayoutMilestone(WebCore::ReachedSessionRestorationRenderTreeSizeThreshold);
    }

    if (auto arguments = std::exchange(m_deferredElementDidFocusArguments, nullptr))
        pageClient().elementDidFocus(arguments->information, arguments->userIsInteracting, arguments->blurPreviousNode, arguments->activityStateChanges, arguments->userData.get());
}

bool WebPageProxy::updateLayoutViewportParameters(const WebKit::RemoteLayerTreeTransaction& layerTreeTransaction)

    m_pendingInputModeChange = WTF::nullopt;

    API::Object* userDataObject = process().transformHandlesToObjects(userData.object()).get();
    if (m_editorState.isMissingPostLayoutData) {
        // FIXME: We should try to eliminate m_deferredElementDidFocusArguments altogether, in favor of only deferring actions that are dependent on post-layout editor state information.
        m_deferredElementDidFocusArguments = makeUnique<ElementDidFocusArguments>(ElementDidFocusArguments { information, userIsInteracting, blurPreviousNode, activityStateChanges, userDataObject });
        return;
    }

    pageClient().elementDidFocus(information, userIsInteracting, blurPreviousNode, activityStateChanges, userDataObject);
}

void WebPageProxy::elementDidBlur()
{
    m_pendingInputModeChange = WTF::nullopt;
    m_deferredElementDidFocusArguments = nullptr;
    pageClient().elementDidBlur();
}




        send(Messages::WebPageProxy::SetEditableElementIsFocused(!element.hasTagName(WebCore::HTMLNames::selectTag)));
#endif
        m_recentlyBlurredElement = nullptr;

        scheduleFullEditorStateUpdate();
    }
}





void WebPage::getFocusedElementInformation(FocusedElementInformation& information)
{
    RefPtr<Document> document = m_page->focusController().focusedOrMainFrame().document();
    if (!document || !document->view())
    if (focusedElement != m_focusedElement)
        return;

    auto focusedElement = m_focusedElement.copyRef();
    bool willLayout = document->view()->needsLayout();
    layoutIfNeeded();

    // Layout may have detached the document or caused a change of focus.
    if (!document->view() || focusedElement != m_focusedElement)
        return;

    if (willLayout)
        sendEditorStateUpdate();
    else
        scheduleFullEditorStateUpdate();

    information.lastInteractionLocation = m_lastInteractionLocation;
    if (auto elementContext = contextForElement(*focusedElement))
        information.elementContext = WTFMove(*elementContext);

Return to Bug 201608

Lines 7537-7544 void WebPageProxy::resetStateAfterProcessExited(ProcessTerminationReason termina a/Source/WebKit/UIProcess/WebPageProxy.cpp_sec1
- a/Source/WebKit/UIProcess/WebPageProxy.cpp -2 lines
7537		7537
7538	#if PLATFORM(IOS_FAMILY)	7538	#if PLATFORM(IOS_FAMILY)
7539	m_waitingForPostLayoutEditorStateUpdateAfterFocusingElement = false;	7539	m_waitingForPostLayoutEditorStateUpdateAfterFocusingElement = false;
7540	m_deferredElementDidFocusArguments = nullptr;
7541
7542	m_isVisibleActivity = nullptr;	7540	m_isVisibleActivity = nullptr;
7543	m_isAudibleActivity = nullptr;	7541	m_isAudibleActivity = nullptr;
7544	m_isCapturingActivity = nullptr;	7542	m_isCapturingActivity = nullptr;

Lines 394-407 typedef GenericCallback<const WebCore::IntPoint&, uint32_t, uint32_t, uint32_t> a/Source/WebKit/UIProcess/WebPageProxy.h_sec1
- a/Source/WebKit/UIProcess/WebPageProxy.h -9 lines
394	typedef GenericCallback<const WebCore::IntPoint&, uint32_t, uint32_t> TouchesCallback;	394	typedef GenericCallback<const WebCore::IntPoint&, uint32_t, uint32_t> TouchesCallback;
395	typedef GenericCallback<const Vector<WebCore::SelectionRect>&> SelectionRectsCallback;	395	typedef GenericCallback<const Vector<WebCore::SelectionRect>&> SelectionRectsCallback;
396	typedef GenericCallback<const FocusedElementInformation&> FocusedElementInformationCallback;	396	typedef GenericCallback<const FocusedElementInformation&> FocusedElementInformationCallback;
397	struct ElementDidFocusArguments {
398	WTF_MAKE_STRUCT_FAST_ALLOCATED;
399	FocusedElementInformation information;
400	bool userIsInteracting;
401	bool blurPreviousNode;
402	OptionSet<WebCore::ActivityState::Flag> activityStateChanges;
403	RefPtr<API::Object> userData;
404	};
405	#endif	397	#endif
406		398
407	#if PLATFORM(COCOA)	399	#if PLATFORM(COCOA)
Lines 2662-2668 private: a/Source/WebKit/UIProcess/WebPageProxy.h_sec2
2662		2654
2663	#if PLATFORM(IOS_FAMILY)	2655	#if PLATFORM(IOS_FAMILY)
2664	Function<bool()> m_deviceOrientationUserPermissionHandlerForTesting;	2656	Function<bool()> m_deviceOrientationUserPermissionHandlerForTesting;
2665	std::unique_ptr<ElementDidFocusArguments> m_deferredElementDidFocusArguments;
2666	bool m_waitingForPostLayoutEditorStateUpdateAfterFocusingElement { false };	2657	bool m_waitingForPostLayoutEditorStateUpdateAfterFocusingElement { false };
2667	WebCore::FloatSize m_maximumUnobscuredSize;	2658	WebCore::FloatSize m_maximumUnobscuredSize;
2668	bool m_lastObservedStateWasBackground { false };	2659	bool m_lastObservedStateWasBackground { false };

Lines 404-412 void WebPageProxy::didCommitLayerTree(const WebKit::RemoteLayerTreeTransaction& a/Source/WebKit/UIProcess/ios/WebPageProxyIOS.mm_sec1
- a/Source/WebKit/UIProcess/ios/WebPageProxyIOS.mm -10 lines
404	m_hitRenderTreeSizeThreshold = true;	404	m_hitRenderTreeSizeThreshold = true;
405	didReachLayoutMilestone(WebCore::ReachedSessionRestorationRenderTreeSizeThreshold);	405	didReachLayoutMilestone(WebCore::ReachedSessionRestorationRenderTreeSizeThreshold);
406	}	406	}
407
408	if (auto arguments = std::exchange(m_deferredElementDidFocusArguments, nullptr))
409	pageClient().elementDidFocus(arguments->information, arguments->userIsInteracting, arguments->blurPreviousNode, arguments->activityStateChanges, arguments->userData.get());
410	}	407	}
411		408
412	bool WebPageProxy::updateLayoutViewportParameters(const WebKit::RemoteLayerTreeTransaction& layerTreeTransaction)	409	bool WebPageProxy::updateLayoutViewportParameters(const WebKit::RemoteLayerTreeTransaction& layerTreeTransaction)
Lines 986-1004 void WebPageProxy::elementDidFocus(const FocusedElementInformation& information, a/Source/WebKit/UIProcess/ios/WebPageProxyIOS.mm_sec2
986	m_pendingInputModeChange = WTF::nullopt;	983	m_pendingInputModeChange = WTF::nullopt;
987		984
988	API::Object* userDataObject = process().transformHandlesToObjects(userData.object()).get();	985	API::Object* userDataObject = process().transformHandlesToObjects(userData.object()).get();
989	if (m_editorState.isMissingPostLayoutData) {
990	// FIXME: We should try to eliminate m_deferredElementDidFocusArguments altogether, in favor of only deferring actions that are dependent on post-layout editor state information.
991	m_deferredElementDidFocusArguments = makeUnique<ElementDidFocusArguments>(ElementDidFocusArguments { information, userIsInteracting, blurPreviousNode, activityStateChanges, userDataObject });
992	return;
993	}
994
995	pageClient().elementDidFocus(information, userIsInteracting, blurPreviousNode, activityStateChanges, userDataObject);	986	pageClient().elementDidFocus(information, userIsInteracting, blurPreviousNode, activityStateChanges, userDataObject);
996	}	987	}
997		988
998	void WebPageProxy::elementDidBlur()	989	void WebPageProxy::elementDidBlur()
999	{	990	{
1000	m_pendingInputModeChange = WTF::nullopt;	991	m_pendingInputModeChange = WTF::nullopt;
1001	m_deferredElementDidFocusArguments = nullptr;
1002	pageClient().elementDidBlur();	992	pageClient().elementDidBlur();
1003	}	993	}
1004		994

Lines 5586-5593 void WebPage::elementDidFocus(WebCore::Element& element) a/Source/WebKit/WebProcess/WebPage/WebPage.cpp_sec1
- a/Source/WebKit/WebProcess/WebPage/WebPage.cpp -2 lines
5586	send(Messages::WebPageProxy::SetEditableElementIsFocused(!element.hasTagName(WebCore::HTMLNames::selectTag)));	5586	send(Messages::WebPageProxy::SetEditableElementIsFocused(!element.hasTagName(WebCore::HTMLNames::selectTag)));
5587	#endif	5587	#endif
5588	m_recentlyBlurredElement = nullptr;	5588	m_recentlyBlurredElement = nullptr;
5589
5590	scheduleFullEditorStateUpdate();
5591	}	5589	}
5592	}	5590	}
5593		5591

Lines 3032-3042 void WebPage::focusNextFocusedElement(bool isForward, CallbackID callbackID) a/Source/WebKit/WebProcess/WebPage/ios/WebPageIOS.mm_sec1
- a/Source/WebKit/WebProcess/WebPage/ios/WebPageIOS.mm -3 / +15 lines
3032		3032
3033	void WebPage::getFocusedElementInformation(FocusedElementInformation& information)	3033	void WebPage::getFocusedElementInformation(FocusedElementInformation& information)
3034	{	3034	{
3035	auto focusedElement = m_focusedElement.copyRef();	3035	RefPtr<Document> document = m_page->focusController().focusedOrMainFrame().document();
3036	layoutIfNeeded();	3036	if (!document \|\| !document->view())
3037	if (focusedElement != m_focusedElement)
3038	return;	3037	return;
3039		3038
		3039	auto focusedElement = m_focusedElement.copyRef();
		3040	bool willLayout = document->view()->needsLayout();
		3041	layoutIfNeeded();
		3042
		3043	// Layout may have detached the document or caused a change of focus.
		3044	if (!document->view() \|\| focusedElement != m_focusedElement)
		3045	return;
		3046
		3047	if (willLayout)
		3048	sendEditorStateUpdate();
		3049	else
		3050	scheduleFullEditorStateUpdate();
		3051
3040	information.lastInteractionLocation = m_lastInteractionLocation;	3052	information.lastInteractionLocation = m_lastInteractionLocation;
3041	if (auto elementContext = contextForElement(*focusedElement))	3053	if (auto elementContext = contextForElement(*focusedElement))
3042	information.elementContext = WTFMove(*elementContext);	3054	information.elementContext = WTFMove(*elementContext);

Lines 1-3 a/Source/WebKit/ChangeLog_sec1
- a/Source/WebKit/ChangeLog +47 lines
		1	2020-03-20 Daniel Bates <dabates@apple.com>
		2
		3	Replace "deferred element focus" functionality with alternative solution
		4	https://bugs.webkit.org/show_bug.cgi?id=201608
		5
		6	Reviewed by Wenson Hsieh.
		7
		8	This is a partial revert of r190278. Have the web process perform a layout, if needed, when
		9	computing focus element information and send an editor state update immediately. If layout is
		10	not needed then explicitly schedule a full editor state update.
		11
		12	Currently, fetching focus element information neither sends an editor state update nor
		13	schedules one. As a result, when the web process tells the UI process to focus an element the
		14	UI process may need to defer doing so if the last received update did not include details
		15	that require up-to-date layout (e.g. the bounding rect of the focused element, which is used
		16	to scroll and zoom to center the focused element). The UI process then schedules an async message
		17	to the web process to fetch the full editor state, which will arrive in a layer tree commit message
		18	from the web process. (Note that the UI process schedules this request to ensure the web process
		19	knows that it is waiting for a layer tree commit. The web process can use this info to expedite
		20	a layer tree commit, if needed). This deferral mechanism complicates the element focusing and
		21	defocusing logic in the UI process and prevents fixing <https://bugs.webkit.org/show_bug.cgi?id=199960>.
		22	Instead remove this deferral concept and have the web process ensure that a full editor state
		23	update is sent or will be sent when computing the focus element information.
		24
		25	* UIProcess/WebPageProxy.cpp:
		26	(WebKit::WebPageProxy::resetStateAfterProcessExited):
		27	* UIProcess/WebPageProxy.h:
		28	* UIProcess/ios/WebPageProxyIOS.mm:
		29	(WebKit::WebPageProxy::didCommitLayerTree):
		30	(WebKit::WebPageProxy::elementDidFocus):
		31	(WebKit::WebPageProxy::elementDidBlur):
		32	Remove bookkeeping code to track a deferred focus event or to perform the deferred event
		33	on layer tree commit.
		34
		35	* WebProcess/WebPage/WebPage.cpp:
		36	(WebKit::WebPage::elementDidFocus):
		37	* WebProcess/WebPage/ios/WebPageIOS.mm:
		38	(WebKit::WebPage::getFocusedElementInformation): Save off whether a layout is needed then
		39	tell the page to layout if needed. If a layout was needed then send an editor state update
		40	immediately (it's an async message): this update will be a "full editor state" update that
		41	includes up-to-date layout details. Otherwise, schedule a full editor state update. While I
		42	am here, I updated the code to take out a ref on the focused frame's document before performing
		43	a layout because layout can cause arbitrary JavaScript execution that could detach the document
		44	from its frame view as part of destroying the document. Document destruction is detected by
		45	checking whether the document has been detached from its frame view. If this happens then
		46	bail out as there is no need to get focus element info.
		47
1	2020-03-19 Daniel Bates <dabates@apple.com>	48	2020-03-19 Daniel Bates <dabates@apple.com>
2		49
3	Have insertDictatedTextAsync() take an InsertTextOptions	50	Have insertDictatedTextAsync() take an InsertTextOptions