C/C++ – Page 2 – 0x2B|~0x2B

Qt4学习笔记 (4)

Posted on January 22, 2015 by gonwan — No Comments ↓

This is a repost from my previous Qt learning series, based on Qt 4.3.

今天来说一些琐碎的细节问题:

1. Item-View 类:

    所谓Item-View类就是类似excel的表格啊, windows资源管理器左边的目录树结构啊之类的控件.
    Qt的Item-View架构有点类似Struts的MVC架构, model表示的是数据, view表示的是数据显示. 没有control了, 一个控件还需要什么control么? 不过Qt还是在中间加了一个东西叫做delegate, 代理. delegate的作用就是: 控制当数据model要修改的时候, UI的view应该怎么显示, 是现实一个combobox呢? 还是一个lineeditor这样. 于是整个Qt的架构被称作model/view架构.
    然后举一个例子来说明一些类的关系. Qt对于比如一个list列表控件会提供这样几个类: QListWidget, QListItem, QListView, QXxxModel(比如QStringListModel). 这几个类的关系是: a) QListWidget = QListView + QXxxModel, QListWidget是Qt提供的一个方便类, 可以拿来直接用, 而如果要自定义一些行为的话, 那么要分别继承QXxxModel和QListView类来达到目的(这个做法让我想到了java/eclipse里的jface的用法). b) QListWidget里的每一项叫做QListItem.
    最后还有一个index和role的概念. QXxxModel里的每一项其实都有一个index, 叫做QModelIndex, 实际上就是一个抽象. 注意它是针对一个model的, 跟QXxxItem没有直接的关系. 这个QModelIndex的作用就是, 可以跟据这个index来拿model中的数据. 这里又有一个role角色的概念–这里其实就类似于一个attribute的名字. 比如你要自己实现一个model, 那么你必须实现如下函数:

QVariant data(const QModelIndex &index, int role) const;

1	QVariant data(const QModelIndex &index, int role) const;

这里role是一个枚举, 可以参看Qt的帮助文档.

2. Container 类:

这个东西其实没什么好多说的, 作用跟C++ STL的容器类一样, Qt也提供比如QVector, QLinkedList, QMap之类的容器类.
不过用Qt的容器类有这样一些好处:
a) 它们支持implicitly shared的feature. 好高级的名字是吧..那如果告诉你其实就是copy-on-write, 那么应该能理解了吧(而COW本身就是一个设计模式: flyweight pattern). 我们用STL容器类的时候, 当要把它作为参数传递的时候, 一般都会传引用. 而如果用Qt类的话则不用. Qt会自己帮我们搞定一切, 不会因为copy参数而降低performance. 说到实现的话, 稍微喵了一眼源码, 其实就是一个引用计数(reference count) 的问题, 呵呵.
b) Qt的容器类能更好的跟其它Qt类来进行merge和互操作.
c) 这里还有一个function object, 或者说functor的东西, 其实这是STL里的概念了. 所谓的functor就是一个class重写了operator()操作符, 一边这个类可以当函数来使用(比如qSort()的第三个参数). 它的好处是, 既然是一个类, 便可以绑定额外的数据.
d) 顺便提一句, QString这个类是支持unicode的.

3. Thread 类:

    Qt的thread类的用法跟linux下pthread类用法很像. 我就不列举用法了. 只说一些需要注意的地方, 多线程总是很麻烦的皑皑.
a) 在同一线程中, singal/slot的调用是同步的, 但是在不同线程中就是异步的. 调用线程只是简单的发送一个event, 然后接受线程在自己的event loop中来调用slot.
b) thread-safe的概念(简称ts吧).. ts函数: 不同线程调用这个函数, 而这个函数会操作全局的共享变量, 如果不会出现同步问题, 那么这个函数就是ts的. ts类: 其中所有的函数都是ts的, 那么这个类也是ts的.
c) reentrant, 可重入的概念. 这个东西很高级, 应该说, 一个函数如果是reentrant的话, 那么它就是ts的; 如果它不是ts的话, 那么它就一定不是reentrant的. reentrant的概念跟多线程无关, 它要求即使在同一线程内, 调用同一函数能正常工作.
    举个例子, 比如strtok()函数一定用过, 每次迭代返回下一个token的时候, 参数都一样, 但是返回不一样. 于是strtok()函数肯定是在内部维护了一个static的buffer以便操作. 由此来说, 包含static或者global变量操作的函数一定不是reentrant的, 我们要避免这种情况.
    还有一个问题, 什么情况下ts的函数不是reentrant的呢? 再举例子, 有两个线程分别用t1和t2表示. 它们同时调用一个ts的函数, 这个函数用mutex来互斥共享数据. 假设t1先进入那个mutex, 于是t2进不去了. 这个时候把t1喀嚓掉(不管你用pthread_cancel()还是别的什么). 然后, 问题就来了..t2这个线程永远卡死了=v=…
d) QObject类是reentrant的. 但是有一些注意事项:
i) 如果要创建一个新的QObject, 那么创建必须在它的parent object(指的是包含关系, 而不是继承)的创建线程中进行这个操作.
ii) 在某个线程中创建QObject必须在那个线程中释放. 如果要释放其它线程中的QObject, 那么可以进行deffered delete: 调用QObject::deleteLater()函数.
iii) Timer类, Network类的使用必须在单个线程中, 跨线程调用是不行的.
iv) 由于底层类库的限制, Qt的widget类都不是reentrant的, 而结果就是a线程不能直接调用b线程中某个widget的成员函数. 有2个妥协的方法: 1) 用singal/slot. 2) 用QMetaObject::invokeMethod() (其实就是singal/slot机制的底层实现函数).
    额额, 多线程就是麻烦呀.. 写了那么多…

4. Network 类:

这个没什么好说的其实.
我们知道网络一般都延时, Qt提供了2套函数: 同步和异步, 分别对应block和non-block的情况, 就这些.

5. XML 类:

也没啥好多说的.
Qt提供了3套XML的API(版本4.3为止; 4.4似乎又有新的API):
a) 是最容易理解的, 写什么就是什么, 读什么就是什么, 基于文本操作. 相关的类有QXmlStreamReader, QXmlStreamWriter.
b) 是SAX API, 基于event driven. 没啥好说的, 标准API啊…
c) 不用说也知道了, DOM API. 又是标准…..

6. 其它:

以上介绍了一些Qt常用类. Qt实际上还包含database的支持(mysql, db2, oracle… 还整个包含sqlite), DnD (Drag& Drop), 2D graphics, I/O(这里有一个设计模式, Serializer Pattern, 呵呵) 等一些类, 由于比较简单, 就不说了. 最新的Qt 4.4还包括Webkit, Phonon等新的API. 4.5会加入ODF, XSLT等支持, 4.6则是USB, bluetooth等… 这个东西已经不是简单的GUI tookit了, 会越来越强大…..

Qt4学习笔记 (3)

Posted on January 22, 2015 by gonwan — No Comments ↓

This is a repost from my previous Qt learning series, based on Qt 4.3.

今天说一下Qt的事件处理(event processing)机制.

    我们之前已经用过了, 当一个QPushButton被点击的时候, 它的clicked() signal就会被emit, 这还是一个signal/slot的关系. 那么Qt中的event到底是干嘛的呢?
    书上有这么一句话: As a rule, signals are useful when using a widget, whereas events are useful when implementing a widget. 也就是说一般的话, 调用signal就可以了, 如果要自定义控件, 那么还是需要用到event的. Qt的event相当于系统native的事件的抽象. 比如windows的话就是对应于message.
    还是根据源码来剖析event和signal的关系. 下图是点击了一个QPushButton之后的call stack:

从QApplication::notify()开始, 然后是QPushButton::event()函数, 其中分别调用父类QAbstractButton:event(), QWidget::event(). 最后这个event被dispatch到QAbstractButton::mouseReleaseEvent(), 在这个函数中, 被判定为click了这个QButton, 即调用QAbstractButtonPrivate::click()函数. 最后再在这个函数中调用QAbstractButtonPrivate::emitClicked()函数来emit QAbstractButton::clicked()这个signal.
所以说, event实际上是singal/slot的底层实现.

Qt4学习笔记 (2)

Posted on January 22, 2015 by gonwan — No Comments ↓

This is a repost from my previous Qt learning series, based on Qt 4.3.

之前的笔记: QT4学习笔记 (1). 可以复习一下, 修改了一些错误.
今天主要说一下Qt的实现, 主要是内存方面. 书上没有写, 完全是自己看源码的, 版本4.3.3 commercial.

先来还是那段最简单的Qt代码:

int main(int argc, char *argv[])
{
    QApplication app(argc, argv);
    QLabel *label = new QLabel("Hello Qt!");
    label->show();
    return app.exec();
}

int main(int argc, char *argv[])

{

QApplication app(argc, argv);

QLabel *label = new QLabel("Hello Qt!");

label->show();

return app.exec();

}

吐了没?= =b.. 还是那个new出来的QLabel的问题, Qt到底怎么样来回收的呢? debug的时候, 我直接把断点设在了QObject和QWidget类的析构函数(直接用debug模式, 打开源码设断点运行就行了). 事实证明还不够. 通过看call stack, 最终发现析构的过程从QApplication这个类开始的.
/src/gui/kernel/qapplication.cpp, 析构函数中, 有这么一段:

    // delete widget mapper
    if (QWidgetPrivate::mapper) {
        QWidgetMapper * myMapper = QWidgetPrivate::mapper;
        QWidgetPrivate::mapper = 0;
        for (QWidgetMapper::Iterator it = myMapper->begin(); it != myMapper->end(); ++it) {
            register QWidget *w = *it;
            if (!w->parent())                        // window
                w->destroy(true, true);
        }
        delete myMapper;
    }

// delete widget mapper

if (QWidgetPrivate::mapper) {

QWidgetMapper * myMapper = QWidgetPrivate::mapper;

QWidgetPrivate::mapper = 0;

for (QWidgetMapper::Iterator it = myMapper->begin(); it != myMapper->end(); ++it) {

if (!w->parent()) // window

w->destroy(true, true);

}

delete myMapper;

}

可能有点难懂, 反正大写Q开头的都是类名. 这段意思大概就是QApplication类会维护一个所有运行中QWidget的mapper, 以便析构函数中作相应处理. 这里似乎只看到调用QWidget的destory()函数, 没有释放内存. debug的结果也验证了, QLabel的析构函数没有被调用到. 难道Qt不具备防止memory leak的么?
我们改一下代码, 把heap内存改成stack内存:

int main(int argc, char *argv[])
{
    QApplication app(argc, argv);
    QLabel label("Hello Qt!");
    label.show();
    return app.exec();
}

int main(int argc, char *argv[])

{

QApplication app(argc, argv);

QLabel label("Hello Qt!");

label.show();

return app.exec();

}

这个显然是没有问题的. 那我们应该怎么用Qt, 才能利用Qt自带的memory管理功能, 只管new, 不用自己delete呢? 再看一段代码:

int main(int argc, char *argv[])
{
    QApplication app(argc, argv);
    {
        QLabel *label = new QLabel("Hello!");
        QHBoxLayout *layout = new QHBoxLayout;
        layout->addWidget(label);
        QDialog dialog;
        dialog.setLayout(layout);
        dialog.show();
    }
    return app.exec();
}

int main(int argc, char *argv[])

{

QApplication app(argc, argv);

{

QLabel *label = new QLabel("Hello!");

QHBoxLayout *layout = new QHBoxLayout;

layout->addWidget(label);

QDialog dialog;

dialog.setLayout(layout);

dialog.show();

}

return app.exec();

}

(以上sb代码debug之用, 切勿模仿…)
我们在QDialog上加了一个QLabel, QDialog是在stack上创建的. debug中, 当QDialog对象调用析构函数时, 在/src/gui/kernel/qwidget.cpp中, 会调用QWidget的析构函数. 在QWidget的析构函数中, 有这样一段:

    if (!d->children.isEmpty())
        d->deleteChildren();

1 2	if (!d->children.isEmpty()) d->deleteChildren();

    d是什么东西? 继续跟进, 发现d是一个QObjectPrivate类的对象, 其中的确有delete的操作(废话, 老子依据call stack反推的啊). 于是我们清楚了, Qt的类之间有父子关系(这里指的是包含, 而不是继承关系), 一旦父类对象被被delete的话, 子类也会被相应的delete. 所以, Qt中我们一般都不需要调用delete的. 但是有一个前提: 由于deleteChildren()函数是QObjectPrivate类的函数, 要使以上条件成立的话, 所有的包含关系的类必须都继承自QObject. 这里有一个design pattern, composite, 嗯.
    另外有一点, QObject类没有定义拷贝构造函数, 也就是说一般的赋值都是shadow copy. 其实是有道理的, 当我们copy一个QObject的时候, 新的object到底是继承原来的children还是不继承呢? 呵呵.
    最后我们再来花点时间讨论一下QObject和QObjectPrivate类的关系. 在/src/corelib/global/qglobal.h中, 有以下代码:

#define Q_D(Class) Class##Private * const d = d_func()
#define Q_Q(Class) Class * const q = q_func()

#define Q_DECLARE_PRIVATE(Class) \
    inline Class##Private* d_func() { return reinterpret_cast<Class##Private *>(d_ptr); } \
    inline const Class##Private* d_func() const { return reinterpret_cast<const Class##Private *>(d_ptr); } \
    friend class Class##Private;

#define Q_DECLARE_PUBLIC(Class) \
    inline Class* q_func() { return static_cast<Class *>(q_ptr); } \
    inline const Class* q_func() const { return static_cast<const Class *>(q_ptr); } \
    friend class Class;

#define Q_D(Class) Class##Private * const d = d_func()

#define Q_Q(Class) Class * const q = q_func()

#define Q_DECLARE_PRIVATE(Class) \

inline Class##Private* d_func() { return reinterpret_cast<Class##Private *>(d_ptr); } \

inline const Class##Private* d_func() const { return reinterpret_cast<const Class##Private *>(d_ptr); } \

friend class Class##Private;

#define Q_DECLARE_PUBLIC(Class) \

inline Class* q_func() { return static_cast<Class *>(q_ptr); } \

inline const Class* q_func() const { return static_cast<const Class *>(q_ptr); } \

friend class Class;

额.. 也就是说, 一般QXxx类和QXxxPrivate类互为friend类, 一般来说QXxxPrivate类都是一些底层实现, 不属于对外的Qt API. 而QXxx类则是对外的Qt API. 有比如说有QProcess和QProcessPrivate两个类, 在/src/corelib/io下, 对应4个文件, qprocess.h, qprocess.cpp, qprocess_p.h, qprocess_win.cpp. 前2个是对外的Qt API, *_p.h是QProcessPrivate类头文件, *_win.cpp则是对应的平台相关的实现代码.

好了, 下班了, 明天再写…

Qt4学习笔记 (1)

Posted on January 22, 2015 by gonwan — No Comments ↓

This is a repost from my previous Qt learning series, based on Qt 4.3.

为什么要学这个? 主要是学习其中的design pattern.
丸子买不起书, 所以看的是电子版: C++ GUI Programming with Qt4, 2nd Edition

作为一个跨平台的GUI库, 首先Qt用的design pattern是facade模式: 不管subsystem如何, 对外提供简单统一接口.
跟wxWidgets一样, 每次你new出来的widget其实都是不用自己来delete的, Qt会自己帮你delete掉. 可以参看源码: /src/gui/kernel/qwidgets.cpp 中的析构函数. 注意如果你用MFC的话, memory是要你自己搞定的.
先来看一段最简单的Qt代码:

#include <QApplication>
#include <QLabel>

int main(int argc, char *argv[])
{
    QApplication app(argc, argv);
    QLabel *label = new QLabel("Hello Qt!");
    label->show();
    return app.exec();
}

#include <QApplication>

#include <QLabel>

int main(int argc, char *argv[])

{

QApplication app(argc, argv);

QLabel *label = new QLabel("Hello Qt!");

label->show();

return app.exec();

}

诶.. 你发现既然是GUI的程序, 为什么用的入口函数是main(), 而不是WinMain()? 能看出这个问题你已经有相当的水平了. 其实Qt自己写了一个WinMain(), 在里面调用你的main(). 源码在这里可以找到: /src/winmain/qtmain_win.cpp (windows版本).
接下来再看一个稍微复杂一点的:

#include <QApplication>
#include <QPushButton>

int main(int argc, char *argv[])
{
    QApplication app(argc, argv);
    QPushButton *button = new QPushButton("Quit");
    QObject::connect(button, SIGNAL(clicked()), &app, SLOT(quit()));
    button->show();
    return app.exec();
}

#include <QApplication>

#include <QPushButton>

int main(int argc, char *argv[])

{

QApplication app(argc, argv);

QPushButton *button = new QPushButton("Quit");

QObject::connect(button, SIGNAL(clicked()), &app, SLOT(quit()));

button->show();

return app.exec();

}

    于是你看到了Qt的事件处理机制signal/slot (信号? 插槽? 好x啊–b). 一个signal相当于一个event, slot相当于一个handler. signal和slot可以是多对多的关系. 其实这里包含了两个dessign pattern. observer模式显而易见, 还有一个就是mediator模式. 这样说应该懂了吧, 不懂的话请翻阅我之前写的design pattern系列的文章.
    另外, 可以参考Qt的官方文档: Signals and Slots. 上面大概说了这么几点重要的:
a) signals/slots/emit实际上都是macro(可以看源码). 对于c++ compiler来说, signals–>protected, slots–>nothing, emit–>nothing, 对于moc(这个工具下面会说)来说, 则被另外不同的preprosessor机制预处理.
b) 通过signal/slot可以调用private的slot(这个我没试过..不太好吧).
c) SIGNAL/SLOT宏, 这2个东西很搞, 实际上就是在函数名前加上”1″或者”2″结成新的字符串–b. 所以, 丸子觉得, 这里可能是不安全的, 如果拼错了的话编译器也是检查不出来的.
d) Q_SIGNALS/Q_SLOTS/Q_EMIT宏有特殊意义, 用于第三方的signal/slot机制避免冲突.
e) signal/slot的函数signature必须: *) 相同. *) 或者slot的参数比signal要少.
    最后, 你看到一个QButton自己show()就可以了. 没错, 因为QButton继承QWidget, QWidget都是可以直接show()的, 这点跟其他的GUI Kit不一样.
    这里有2个工具: moc(Meta-Object Compiler), uic(User Interface Compiler).
    moc用来维护Qt中类的运行时信息, 把它想象成Java中的reflection就可以了. 那么麻烦主要是由于C++本身不支持这个. 所以, 实际上signal/slot机制的实现也用到了这些信息, 因此一个Qt类一定要实现某些特定的接口函数, 这就是moc所做的工作了.
    uic用来把*.ui文件转化成C++代码, 以便能编译到一个binary中去. 用xml做ui真的是一个很不错的想法, 不过这里有一个问题啊. 那就是用uic生成C++代码之前, 其它C++类如果要引用ui中的成员变量似乎是不可能的. 诶.. 这个不知道怎么解决.

つつく.

std::thread and std::future in C++11

Posted on December 12, 2014 by gonwan — No Comments ↓

This is a quick note to chapter 4 of C++ Concurrency in Action.

1. std::thread

In C++11, It’s quite simple to create a separate thread using std::thread. Following code will simply output “hello world” or “world hello”:

#include <iostream>
#include <thread>
using namespace std;

void foo(const char *s) {
    cout << s << endl;
}

int main() {
    thread t(foo, "hello");
    cout << "world" << endl;
    /* destructor of std::thread calls std::terminate(), so we should call join() manually. */
    t.join();
    return 0;
}

#include <iostream>

#include <thread>

using namespace std;

void foo(const char *s) {

cout << s << endl;

}

int main() {

thread t(foo, "hello");

cout << "world" << endl;

/* destructor of std::thread calls std::terminate(), so we should call join() manually. */

t.join();

return 0;

}

2. std::mutex and std::condition_variable

If you need synchronization between threads, there are std::mutex and std::condition_variable. The semantics are the same with that in pthread library. Here’s a simple producer/consumer demo:

#include <iostream>
#include <chrono>
#include <condition_variable>
#include <mutex>
#include <queue>
#include <thread>
using namespace std;

queue<int> q;
mutex m;
condition_variable c;
const chrono::milliseconds ms(1000);

void producer() {
    static int i = 0;
    while (true) {
        m.lock();
        cout << "pushing " << i << endl;
        q.push(i++);
        c.notify_one();
        m.unlock();
        this_thread::sleep_for(ms);
    }
}

void consumer() {
    while (true) {
        unique_lock<mutex> lk(m);
        c.wait(lk, [](){ return !q.empty(); });
        int i = q.front();
        cout << "popping " << i << endl;
        q.pop();
        lk.unlock();
        this_thread::sleep_for(ms);
    }
}

int main() {
    thread t(consumer);
    thread t2(producer);
    t.join();
    t2.join();
    return 0;
}

#include <iostream>

#include <chrono>

#include <condition_variable>

#include <mutex>

#include <queue>

#include <thread>

using namespace std;

queue<int> q;

mutex m;

condition_variable c;

const chrono::milliseconds ms(1000);

void producer() {

static int i = 0;

while (true) {

m.lock();

cout << "pushing " << i << endl;

q.push(i++);

c.notify_one();

m.unlock();

this_thread::sleep_for(ms);

}

void consumer() {

while (true) {

unique_lock<mutex> lk(m);

c.wait(lk, [](){ return !q.empty(); });

int i = q.front();

cout << "popping " << i << endl;

q.pop();

lk.unlock();

this_thread::sleep_for(ms);

}

int main() {

thread t(consumer);

thread t2(producer);

t.join();

t2.join();

return 0;

}

3. std::future with std::async()

C++11 also simplifies our work with one-off events with std::future. std::future provides a mechanism to access the result of asynchronous operations. It can be used with std::async(), std::packaged_task and std::promise. Starting with std::async():

#include <iostream>
#include <future>
using namespace std;

void foo(const char *s) {
    cout << s << endl;
}

int bar(int a, int b) {
    return a + b;
}

int main() {
    /* auto will be simpler */
    future<void> f = std::async(foo, "hello");
    future<int> f2 = std::async(launch::async, bar, 1, 2);
    /* f.get() is required if f is deferred by the library */
    //f.get();
    /* std::async() can return a value */
    cout << "1 + 2 = " << f2.get() << endl;
    /* threads created by std::async() are joined automatically */
    return 0;
}

#include <iostream>

#include <future>

using namespace std;

void foo(const char *s) {

cout << s << endl;

}

int bar(int a, int b) {

return a + b;

}

int main() {

/* auto will be simpler */

future<void> f = std::async(foo, "hello");

future<int> f2 = std::async(launch::async, bar, 1, 2);

/* f.get() is required if f is deferred by the library */

//f.get();

/* std::async() can return a value */

cout << "1 + 2 = " << f2.get() << endl;

/* threads created by std::async() are joined automatically */

return 0;

}

std::async() gives two advantages over the direct usage of std::thread. Threads created by it are automatically joined. And we can now have a return value. std::async() decides whether to run the callback function in a separate thread or just in the current thread. But there’s a chance to specify a control flag(launch::async or launch::deferred) to tell the library, what approach we want it to run the callback.

When testing With gcc-4.8, foo() is not called. But with VC++2013, it does output “hello”.

4. std::future with std::packaged_task

With std::async(), we cannot control when our callback function is invoked. That’s what std::packaged_task is designed to deal with. It’s just a wrapper to callables. We can request an associated std::future from it. And when a std::packaged_task is invoked and finished, the associated future will be ready:

#include <iostream>
#include <future>
using namespace std;

void foo() {
    cout << "in pt.." << endl;
}

int bar(int a, int b) {
    cout << "in pt2.." << endl;
    return a + b;
}

/* associate with tasks */
packaged_task<void()> pt(foo);
packaged_task<int(int,int)> pt2(bar);

void waiter() {
    /* get associated future */
    auto f = pt.get_future();
    /* wait here */
    f.get();
    cout << "after f.get().." << endl;
}

void waiter2() {
    auto f2 = pt2.get_future();
    f2.get();
    cout << "after f2.get().." << endl;
}

int main() {
    auto t = std::async(launch::async, waiter);
    auto t2 = std::async(launch::async, waiter2);
    /* associated futures will be ready when the packaged tasks complete */
    pt();
    pt2(1, 2);
    return 0;
}

#include <iostream>

#include <future>

using namespace std;

void foo() {

cout << "in pt.." << endl;

}

int bar(int a, int b) {

cout << "in pt2.." << endl;

return a + b;

}

/* associate with tasks */

packaged_task<void()> pt(foo);

packaged_task<int(int,int)> pt2(bar);

void waiter() {

/* get associated future */

auto f = pt.get_future();

/* wait here */

f.get();

cout << "after f.get().." << endl;

}

void waiter2() {

auto f2 = pt2.get_future();

f2.get();

cout << "after f2.get().." << endl;

}

int main() {

auto t = std::async(launch::async, waiter);

auto t2 = std::async(launch::async, waiter2);

/* associated futures will be ready when the packaged tasks complete */

pt();

pt2(1, 2);

return 0;

}

In waiter() and waiter2(), future::get() blocks until the associating std::packaged_task completes. You will always get “in pt” before “after f.get()” and “in pt2” before “after f2.get()”. They are synchronized.

5. std::future with std::promise

You may also need to get notified in the middle of a task. std::promise can help you. It works like a lightweight event.

Future and Promise are the two separate sides of an asynchronous operation. std::promise is used by the “producer/writer”, while std::future is used by the “consumer/reader”. The reason it is separated into these two interfaces is to hide the “write/set” functionality from the “consumer/reader”:

#include <iostream>
#include <future>
using namespace std;

promise<bool> p;
promise<int> p2;

void waiter() {
    /* get associated future */
    auto f = p.get_future();
    /* wait here */
    f.get();
    cout << "after f.get().." << endl;
}

void waiter2() {
    auto f2 = p2.get_future();
    try {
        f2.get();
    } catch (...) {
        /* caught exception */
        cout << "caught exception in f2.get().." << endl;
    }
    cout << "after f2.get().." << endl;
}

int main() {
    auto t = std::async(launch::async, waiter);
    auto t2 = std::async(launch::async, waiter2);
    /* associated futures will be ready after a value is set */
    cout << "setting p.." << endl;
    p.set_value(true);
    /* exceptions can also be set */
    cout << "setting p2.." << endl;
    p2.set_exception(std::exception_ptr(nullptr));
    return 0;
}

#include <iostream>

#include <future>

using namespace std;

promise<bool> p;

promise<int> p2;

void waiter() {

/* get associated future */

auto f = p.get_future();

/* wait here */

f.get();

cout << "after f.get().." << endl;

}

void waiter2() {

auto f2 = p2.get_future();

try {

f2.get();

} catch (...) {

/* caught exception */

cout << "caught exception in f2.get().." << endl;

}

cout << "after f2.get().." << endl;

}

int main() {

auto t = std::async(launch::async, waiter);

auto t2 = std::async(launch::async, waiter2);

/* associated futures will be ready after a value is set */

cout << "setting p.." << endl;

p.set_value(true);

/* exceptions can also be set */

cout << "setting p2.." << endl;

p2.set_exception(std::exception_ptr(nullptr));

return 0;

}

Again in waiter() and waiter2(), future::get() blocks until a value or an exception is set into the associating std::promise. So “setting p” is always before “f.get()” and “setting p2” is always before “f2.get()”. They are synchronized.

NOTE: std::future seems to be not correctly implemented in VC++2013. So the last two code snippet do not work with it. But you can try the online VC++2015 compiler(still in preview as this writing), it works.

Two-phase Lookup in C++ Templates

Posted on December 12, 2014 by gonwan — No Comments ↓

This is a quick note to C++ Templates: The Complete Guide. Name Taxonomy comes first in chapter 9:

Qualified name: This term is not defined in the standard, but we use it to refer to names that undergo so-called qualified lookup. Specifically, this is a qualified-id or an unqualified-id that is used after an explicit member access operator (. or ->). Examples are S::x, this->f, and p->A::m. However, just class_mem in a context that is implicitly equivalent to this->class_mem is not a qualified name: The member access must be explicit.

Unqualified name: An unqualified-id that is not a qualified name. This is not a standard term but corresponds to names that undergo what the standard calls unqualified lookup.

Dependent name: A name that depends in some way on a template parameter. Certainly any qualified or unqualified name that explicitly contains a template parameter is dependent. Furthermore, a qualified name that is qualified by a member access operator (. or ->) is dependent if the type of the expression on the left of the access operator depends on a template parameter. In particular, b in this->b is a dependent name when it appears in a template. Finally, the identifier ident in a call of the form ident(x, y, z) is a dependent name if and only if any of the argument expressions has a type that depends on a template parameter.

Nondependent name: A name that is not a dependent name by the above description.

And the definition from Chapter 10:

This leads to the concept of two-phase lookup: The first phase is the parsing of a template, and the second phase is its instantiation.

During the first phase, nondependent names are looked up while the template is being parsed using both the ordinary lookup rules and, if applicable, the rules for argument-dependent lookup (ADL). Unqualified dependent names (which are dependent because they look like the name of a function in a function call with dependent arguments) are also looked up that way, but the result of the lookup is not considered complete until an additional lookup is performed when the template is instantiated.

During the second phase, which occurs when templates are instantiated at a point called the point of instantiation(POI), dependent qualified names are looked up (with the template parameters replaced with the template arguments for that specific instantiation), and an additional ADL is performed for the unqualified dependent names.

To summarize: nondependent names are looked up in first phase, qualified dependent names are looked up in second phase, and unqualified dependent names are looked up in both phases. Some code to illustrate how this works:

#include <iostream>

template <typename T>
struct Base {
    typedef int I;
};

template <typename T>
struct Derived : Base<T> {
    void foo() {
        //typename Base<T>::I i = 1.024;
        I i = 1.024;
        std::cout << i << std::endl;
    }
};

template <>
struct Base<void> {
    //const static int I = 0;
    typedef double I;
};

int main() {
    Derived<bool> d1;
    d1.foo();
    Derived<void> d2;
    d2.foo();
    return 0;
}

#include <iostream>

template <typename T>

struct Base {

typedef int I;

};

template <typename T>

struct Derived : Base<T> {

void foo() {

//typename Base<T>::I i = 1.024;

I i = 1.024;

std::cout << i << std::endl;

}

};

template <>

struct Base<void> {

//const static int I = 0;

typedef double I;

};

int main() {

Derived<bool> d1;

d1.foo();

Derived<void> d2;

d2.foo();

return 0;

}

Now look into Derived::foo(). I is a nondependent name, it should be looked up only in first phase. But at that point, the compiler cannot decide the type of it. When instantiated with Derived<bool>, I is type int. When instantiated with Derived<void>, I is type double. So it’s better to look up I in the second phase. We can use typename Base<T>::I i = 1.024; to delay the look up, for I is a qualified dependent name now.

Unfortunately, two-phase lookup(C++03 standard) is not fully supported in VC++ even in VC++2013. It compiles well and gives your most expecting result(output 1 and 1.024). With gcc-4.6, it gives errors like:

temp1.cpp: In member function ‘void Derived<T>::foo()’:
temp1.cpp:12:9: error: ‘I’ was not declared in this scope
temp1.cpp:12:11: error: expected ‘;’ before ‘i’
temp1.cpp:13:22: error: ‘i’ was not declared in this scope

temp1.cpp: In member function ‘void Derived<T>::foo()’:

temp1.cpp:12:9: error: ‘I’ was not declared in this scope

temp1.cpp:12:11: error: expected ‘;’ before ‘i’

temp1.cpp:13:22: error: ‘i’ was not declared in this scope

Another code snippet:

#ifdef _USE_STRUCT
/* ADL of nondependent names in two-phase lookup should
 * only works for types that have an associated namespace. */
struct Int { 
    Int(int) { };
};
#else
typedef int Int;
#endif

template <typename T>
void f(T i) {
    g(i);
};

void g(Int i) {
}

int main() {
    f(Int(1024));
    return 0;
}

#ifdef _USE_STRUCT

/* ADL of nondependent names in two-phase lookup should

* only works for types that have an associated namespace. */

struct Int {

Int(int) { };

};

#else

typedef int Int;

#endif

template <typename T>

void f(T i) {

g(i);

};

void g(Int i) {

}

int main() {

f(Int(1024));

return 0;

}

When the compiler sees f(), g() has not been declared. This code should not compile, if f() is a nontemplate function. Since f() is a template function and g() is a nondependent name, the compiler can use ADL in first phase to find the declaration of g(). Note, a user-defined type like Int is required here. Since int is a primitive type, it has no associated namespace, and no ADL is performed.

VC++2013 still compiles well with this code. You can find some clue that they will not support it in the next VC++2015 release. With gcc, they declared to fully support two-phase lookup in gcc-4.7. I used gcc-4.8, error output looks like:

temp2.cpp: In instantiation of ‘void f(T) [with T = int]’:
temp2.cpp:20:16:   required from here
temp2.cpp:13:8: error: ‘g’ was not declared in this scope, and no declarations were found by argument-dependent lookup at the point of instantiation [-fpermissive]
     g(i);
        ^
temp2.cpp:16:6: note: ‘void g(Int)’ declared here, later in the translation unit
 void g(Int i) {
      ^

temp2.cpp: In instantiation of ‘void f(T) [with T = int]’:

temp2.cpp:20:16: required from here

temp2.cpp:13:8: error: ‘g’ was not declared in this scope, and no declarations were found by argument-dependent lookup at the point of instantiation [-fpermissive]

g(i);

temp2.cpp:16:6: note: ‘void g(Int)’ declared here, later in the translation unit

void g(Int i) {

And the code compiles well with self-defined type Int(using -D_USE_STRUCT switch).

Different Semantics Between Win32 Events and Condition Variables

Posted on November 26, 2014 by gonwan — No Comments ↓

Following the last post, I’m trying to implement a thread pool for practise, which supposed to work under both Windows and Linux platform. But the different semantics between Win32 events and condition variables makes it impossible to code in a unified logic. First, Linux uses mutex and condition variable to keep synchronization. While there is only event under Windows. Then, pthread_cond_signal() does nothing if no thread is currently waiting on the condition:

int main() {
    pthread_mutex_t m;
    pthread_cond_t c;
    pthread_mutex_init(&m, NULL);
    pthread_cond_init(&c, NULL);
    /* signal & wait */
    pthread_cond_signal(&c);
    pthread_mutex_lock(&m); /* hangs here, since signal is missed. */
    pthread_cond_wait(&c, &m);
    pthread_mutex_unlock(&m);
    /* destroy */
    pthread_mutex_destroy(&m);
    pthread_cond_destroy(&c);
    return 0;
}

int main() {

pthread_mutex_t m;

pthread_cond_t c;

pthread_mutex_init(&m, NULL);

pthread_cond_init(&c, NULL);

/* signal & wait */

pthread_cond_signal(&c);

pthread_mutex_lock(&m); /* hangs here, since signal is missed. */

pthread_cond_wait(&c, &m);

pthread_mutex_unlock(&m);

/* destroy */

pthread_mutex_destroy(&m);

pthread_cond_destroy(&c);

return 0;

}

But under Windows, code below simply pass through:

int main() {
    HANDLE e = CreateEvent(NULL, FALSE, FALSE, NULL);
    SetEvent(e);
    WaitForSingleObject(e, INFINITE); /* won't miss. */
    return 0;
}

int main() {

HANDLE e = CreateEvent(NULL, FALSE, FALSE, NULL);

SetEvent(e);

WaitForSingleObject(e, INFINITE); /* won't miss. */

return 0;

}

And, under Windows Vista and later versions, a new series of synchronization API was introduced to align with the Linux API:

int main() { /* requires windows vista and later. */
    CRITICAL_SECTION cs;
    CONDITION_VARIABLE cv;
    InitializeCriticalSection(&cs);
    InitializeConditionVariable(&cv);
    /* signal & wait */
    WakeConditionVariable(&cv);
    EnterCriticalSection(&cs);
    SleepConditionVariableCS(&cv, &cs, INFINITE); /* hangs here, since signal is missed. */
    LeaveCriticalSection(&cs);
    /* destroy */
    DeleteCriticalSection(&cs);
    return 0;
}

int main() { /* requires windows vista and later. */

CRITICAL_SECTION cs;

CONDITION_VARIABLE cv;

InitializeCriticalSection(&cs);

InitializeConditionVariable(&cv);

/* signal & wait */

WakeConditionVariable(&cv);

EnterCriticalSection(&cs);

SleepConditionVariableCS(&cv, &cs, INFINITE); /* hangs here, since signal is missed. */

LeaveCriticalSection(&cs);

/* destroy */

DeleteCriticalSection(&cs);

return 0;

}

Spurious Wakeups

Posted on November 20, 2014 by gonwan — No Comments ↓

http://vladimir_prus.blogspot.com/2005/07/spurious-wakeups.html

One of the two basic synchronisation primitives in multithreaded programming is called “condition variables”. Here’s a small example:

bool something_happened;
boost::mutex m;
boost::condition_variable c;
void thread1() {
    boost::mutex::scoped_lock(m);
    while (!something_happened) {
        c.wait(m);
    }
}
void thread2() {
    // do lots of work
    boost::mutex::scoped_lock(m);
    something_happened = true;
    c.notify_one();
}

bool something_happened;

boost::mutex m;

boost::condition_variable c;

void thread1() {

boost::mutex::scoped_lock(m);

while (!something_happened) {

c.wait(m);

}

void thread2() {

// do lots of work

boost::mutex::scoped_lock(m);

something_happened = true;

c.notify_one();

}

Here, the call to “c.wait()” unlocks the mutex (allowing the other thread to eventually lock it), and suspends the calling thread. When another thread calls ‘notify’, the first thread wakes up, locks the mutex again (implicitly, inside ‘wait’), sees that variable is set to ‘true’ and goes on.

But why do we need the while loop, can’t we write:

    if (!something_happened) {
        c.wait(m);
    }

if (!something_happened) {

c.wait(m);

}

We can’t. And the killer reason is that ‘wait’ can return without any ‘notify’ call. That’s called spurious wakeup and is explicitly allowed by POSIX. Essentially, return from ‘wait’ only indicates that the shared data might have changed, so that data must be evaluated again.

Okay, so why this is not fixed yet? The first reason is that nobody wants to fix it. Wrapping call to ‘wait’ in a loop is very desired for several other reasons. But those reasons require explanation, while spurious wakeup is a hammer that can be applied to any first year student without fail.

The second reason is that fixing this is supposed to be hard. Most sources I’ve seen say that fixing that would require very large overhead on certain architectures. Strangely, no details were ever given, which made me wonder if avoiding spurious wakeups is simple, but all the threading experts secretly decided to tell everybody it’s hard.

After asking on comp.programming.thread, I at least know the reason for Linux (thanks to Ben Hutchings). Internally, wait is implemented as a call to the ‘futex’ system call. Each blocking system call on Linux returns abruptly when the process receives a signal — because calling signal handler from kernel call is tricky. What if the signal handler calls some other system function? And a new signal arrives? It’s easy to run out of kernel stack for a process. Exactly because each system call can be interrupted, when glibc calls any blocking function, like ‘read’, it does it in a loop, and if ‘read’ returns EINTR, calls ‘read’ again.

Can the same trick be used to conditions? No, because the moment we return from ‘futex’ call, another thread can send us notification. And since we’re not waiting inside ‘futex’, we’ll miss the notification(A third thread can get it, and change the value of predicate. — gonwan). So, we need to return to the caller, and have it reevaluate the predicate. If another thread indeed set it to true, we’ll break out of the loop.

So much for spurious wakeups on Linux. But I’m still very interested to know what the original reasons were.

==============================
Also see the explanation for spurious wakeups on the linux man page: pthread_cond_signal.
Last note: PulseEvent() in windows(manual-reset) = pthread_cond_signal() in linux, while SetEvent() in windows(auto-reset) = pthread_cond_broadcast() in linux, see here and here. And spurious wakeups are also possible on windows when using condition variables.

C++ Template Function Overload & Specialization

Posted on July 6, 2014 by gonwan — No Comments ↓

Currently reading C++ Templates: The Complete Guide these days. Just summarize confusions about template function overload & specialization.

#include <iostream>
using namespace std;

template <typename T>
void foo(T) {
    cout << "in foo(T)" << endl;    
}

/* overload only */
template <typename T>
void foo(T *) {
    cout << "in foo(T *)" << endl;
}

/* full specialization of foo(T *) */
template <>
void foo<int>(int *) {
    cout << "in foo(int *)" << endl;
}

int main()
{
    /* The compiler does overload resolution before it looks at specializations. */
    foo(new int);
    return 0;
}

#include <iostream>

using namespace std;

template <typename T>

void foo(T) {

cout << "in foo(T)" << endl;

}

/* overload only */

template <typename T>

void foo(T *) {

cout << "in foo(T *)" << endl;

}

/* full specialization of foo(T *) */

template <>

void foo<int>(int *) {

cout << "in foo(int *)" << endl;

}

int main()

{

/* The compiler does overload resolution before it looks at specializations. */

foo(new int);

return 0;

}

It first selects foo(T *) over foo(T), and use the specialized version foo<int>(int *).

#include <iostream>
using namespace std;

template <typename T>
void foo(T) {
    cout << "in foo(T)" << endl;    
}

/* overload only */
template <typename T>
void foo(T *) {
    cout << "in foo(T *)" << endl;
}

/* full specialization of foo(T) */
template <>
void foo<int *>(int *) {
    cout << "in foo(int *)" << endl;
}

#if 0
/* function partial specialization, not allowed in c++ specification. */
template <typename T>
void foo<T *>(T *) {
    cout << "in foo<T *>(T *)" << endl;
}
#endif

int main()
{
    /* The compiler does overload resolution before it looks at specializations. */
    foo(new int);
    return 0;
}

#include <iostream>

using namespace std;

template <typename T>

void foo(T) {

cout << "in foo(T)" << endl;

}

/* overload only */

template <typename T>

void foo(T *) {

cout << "in foo(T *)" << endl;

}

/* full specialization of foo(T) */

template <>

void foo<int *>(int *) {

cout << "in foo(int *)" << endl;

}

#if 0

/* function partial specialization, not allowed in c++ specification. */

template <typename T>

void foo<T *>(T *) {

cout << "in foo<T *>(T *)" << endl;

}

#endif

int main()

{

/* The compiler does overload resolution before it looks at specializations. */

foo(new int);

return 0;

}

Here it just selects foo(T *). The foo<int *>(int *) version is not a specialization of it, and is not selected.

Note the syntax of function partial specialization(if possible) here.

Difference between overload and specilization from the book:

In Chapter 12 we discussed how class templates can be partially specialized, whereas function templates are simply overloaded. The two mechanisms are somewhat different.

Partial specialization doesn’t introduce a completely new template: It is an extension of an existing template (the primary template). When a class template is looked up, only primary templates are considered at first. If, after the selection of a primary template, it turns out that there is a partial specialization of that template with a template argument pattern that matches that of the instantiation, its definition (in other words, its body) is instantiated instead of the definition of the primary template. (Full template specializations work exactly the same way.)

In contrast, overloaded function templates are separate templates that are completely independent of one another. When selecting which template to instantiate, all the overloaded templates are considered together, and overload resolution attempts to choose one as the best fit. At first this might seem like an adequate alternative, but in practice there are a number of limitations:

– It is possible to specialize member templates of a class without changing the definition of that class. However, adding an overloaded member does require a change in the definition of a class. In many cases this is not an option because we may not own the rights to do so. Furthermore, the C++ standard does not currently allow us to add new templates to the std namespace, but it does allow us to specialize templates from that namespace.

– To overload function templates, their function parameters must differ in some material way. Consider a function template R convert(T const&) where R and T are template parameters. We may very well want to specialize this template for R = void, but this cannot be done using overloading.

– Code that is valid for a nonoverloaded function may no longer be valid when the function is overloaded. Specifically, given two function templates f(T) and g(T) (where T is a template parameter), the expression g(&f) is valid only if f is not overloaded (otherwise, there is no way to decide which f is meant).

– Friend declarations refer to a specific function template or an instantiation of a specific function template. An overloaded version of a function template would not automatically have the privileges granted to the original template.

Setting up Pretty Printers in GDB

Posted on July 2, 2014 by gonwan — No Comments ↓

There’s an official page on gcc website: https://sourceware.org/gdb/wiki/STLSupport. And here is how to set up it under Ubuntu.

Under Ubuntu 12.04(Precise), simply install the libstdc++6 debug package. It includes the python script for pretty printers:

# sudo apt-get install libstdc++6-4.6-dbg

1	# sudo apt-get install libstdc++6-4.6-dbg

Create a .gdbinit file in your home directory, with the content:

python
import sys
sys.path.insert(0, '/usr/share/gcc-4.6/python')
from libstdcxx.v6.printers import register_libstdcxx_printers
register_libstdcxx_printers (None)
end

python

import sys

sys.path.insert(0, '/usr/share/gcc-4.6/python')

from libstdcxx.v6.printers import register_libstdcxx_printers

register_libstdcxx_printers (None)

end

My test program looks like:

#include <string>
#include <vector>
using namespace std;

int main()
{
    string s = "test";
    vector<int> vi;
    vi.push_back(1);
    vi.push_back(2);
    vi.push_back(3);
    return 0;
}

#include <string>

#include <vector>

using namespace std;

int main()

{

string s = "test";

vector<int> vi;

vi.push_back(1);

vi.push_back(2);

vi.push_back(3);

return 0;

}

Build it with debugging enabled(-g):

# g++ -g test.cpp -o test

1	# g++ -g test.cpp -o test

Debug it with GDB:

# gdb -q test 
Reading symbols from /home/gonwan/test...done.
(gdb) b 12
Breakpoint 1 at 0x8048842: file test.cpp, line 12.
(gdb) r
Starting program: /home/gonwan/test 

Breakpoint 1, main () at test.cpp:12
12	    return 0;
(gdb) p s
$1 = "test"
(gdb) p vi
$2 = std::vector of length 3, capacity 4 = {1, 2, 3}
(gdb)

# gdb -q test

Reading symbols from /home/gonwan/test...done.

(gdb) b 12

Breakpoint 1 at 0x8048842: file test.cpp, line 12.

(gdb) r

Starting program: /home/gonwan/test

Breakpoint 1, main () at test.cpp:12

12 return 0;

(gdb) p s

$1 = "test"

(gdb) p vi

$2 = std::vector of length 3, capacity 4 = {1, 2, 3}

(gdb)

Without pretty printers, the output is tedious and hard to understand:

$ gdb -q test 
Reading symbols from /home/gonwan/test...done.
(gdb) set print pretty
(gdb) b 12
Breakpoint 1 at 0x8048842: file test.cpp, line 12.
(gdb) r
Starting program: /home/gonwan/test 

Breakpoint 1, main () at test.cpp:12
12	    return 0;
(gdb) p s
$1 = {
  static npos = <optimized out>, 
  _M_dataplus = {
    <std::allocator<char>> = {
      <__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, 
    members of std::basic_string<char, std::char_traits<char>, std::allocator<char> >::_Alloc_hider: 
    _M_p = 0x804c014 "test"
  }
}
(gdb) p vi
$2 = {
  <std::_Vector_base<int, std::allocator<int> >> = {
    _M_impl = {
      <std::allocator<int>> = {
        <__gnu_cxx::new_allocator<int>> = {<No data fields>}, <No data fields>}, 
      members of std::_Vector_base<int, std::allocator<int> >::_Vector_impl: 
      _M_start = 0x804c040, 
      _M_finish = 0x804c04c, 
      _M_end_of_storage = 0x804c050
    }
  }, <No data fields>}
(gdb)

$ gdb -q test

Reading symbols from /home/gonwan/test...done.

(gdb) set print pretty

(gdb) b 12

Breakpoint 1 at 0x8048842: file test.cpp, line 12.

(gdb) r

Starting program: /home/gonwan/test

Breakpoint 1, main () at test.cpp:12

12 return 0;

(gdb) p s

$1 = {

static npos = <optimized out>,

_M_dataplus = {

<std::allocator<char>> = {

<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>},

members of std::basic_string<char, std::char_traits<char>, std::allocator<char> >::_Alloc_hider:

_M_p = 0x804c014 "test"

}

(gdb) p vi

$2 = {

<std::_Vector_base<int, std::allocator<int> >> = {

_M_impl = {

<std::allocator<int>> = {

<__gnu_cxx::new_allocator<int>> = {<No data fields>}, <No data fields>},

members of std::_Vector_base<int, std::allocator<int> >::_Vector_impl:

_M_start = 0x804c040,

_M_finish = 0x804c04c,

_M_end_of_storage = 0x804c050

}

}, <No data fields>}

(gdb)

Under Ubuntu 14.04(Trusty), the 4.8 version of debug package should be installed:

# sudo apt-get install libstdc++6-4.8-dbg

1	# sudo apt-get install libstdc++6-4.8-dbg

There’s an additional step. Since GDB in Trusty is built with python3, not python2, and the python scripts for pretty printers are in python2 syntax. A simple conversion is required:

# sudo 2to3 -w /usr/share/gcc-4.8/python/libstdcxx/v6/printers.py

1	# sudo 2to3 -w /usr/share/gcc-4.8/python/libstdcxx/v6/printers.py

Backup it before conversion if neccessary.